You added retrieval and the error rate barely moved. Di...

If retrieval does not improve performance, you must isolate the failure points. First, measure answer-passage recall to determine if the relevant information is actually being retrieved. Second, perform a closed-book ablation to see how the model performs without retrieval, which helps isolate the contribution of the model’s internal weights.

If recall is high but errors persist, the issue may be that retrieval is displacing correct parametric knowledge (high phi) or that the errors are inherent to the model’s reconciliation process (causes 2 and 5), where retrieval cannot help.