← All questions
RAGHard· Query Rewriting· Cost Analysis· System Design

For a corpus of 10 million documents serving 2 million...

To decide between query-time rewriting and document pre-expansion, you must derive the break-even point. With 2 million queries per day, the cumulative cost of running an LLM for every query will overtake the one-time cost of indexing the 10 million documents within a few days. While query-time rewriting has the advantage of adapting to the specific nuances of a live user query, the sheer volume of traffic makes it the more expensive architectural choice compared to the static, one-time cost of pre-expanding the document corpus.

Code playground (Python)