← All questions
EvaluationVery Hard· RAG Evaluation· Recall· Multi-hop Reasoning· System Debugging

Retrieval owns recall@5 of 0.94 and says the generator...

The current metrics are misleading. Recall@5 measures topical relevance to the surface query, which makes it silent on critical failure modes such as multi-hop reasoning errors or information staleness. Because the p95 budget of 800 ms prevents adding additional round-trips to the query process, the teams must stop relying on surface-level metrics. The resolution requires updating the evaluation set to specifically annotate evidence chains and timestamps to determine if the retrieval or the generation is truly at fault.

Code playground (Python)