← All questions
RAGHard· Retrieval· System Design· Evaluation

Our datastore just grew from 2M to 200M chunks. Can we...

The assumption that a larger datastore makes the generator’s job easier is incorrect. A hundredfold increase in chunks makes the top-k window more complex due to the higher likelihood of encountering contradictory or near-duplicate information. Consequently, the generator’s comprehension burden increases. The correct approach is to measure performance by holding the retrieval window (k) constant and evaluating grounded accuracy and conflict resolution on both the 70B and 8B models to determine if the performance delta is acceptable.

Code playground (Python)