← All questions
Retrieval & EmbeddingsMedium· System Design· Cost Optimization· Scalability

For a large corpus, should you use document-side query...

The choice depends on the break-even point between indexing costs and per-query inference costs. For a corpus of 10 million documents and 2 million queries per day, the cumulative query volume will quickly surpass the one-time indexing cost, making per-query LLM rewriting expensive. However, query-time rewriting is superior for adapting to live queries that the document author never anticipated. A robust system design often employs both: document-side expansion for baseline coverage and query-time rewriting for specific, complex user intent.

Code playground (Python)