← All questions
RAGHard· Retrieval· In-Context Learning· Evaluation· Debugging

Your RAG assistant scores well on the offline set and d...

When a RAG system performs well offline but degrades in production, the first step is to isolate the source of the error. You should check whether the gold passage exists within the assembled context provided to the model. If the gold passage is absent, the issue is strictly a retrieval failure, and no amount of prompt engineering or ICL adjustments will resolve it. If the gold passage is present, the issue lies in the ICL process. In this case, you should hold the context fixed and vary the demonstrations to improve the model’s performance.

Code playground (Python)