After a prompt change, faithfulness rose from 0.70 to 0...

When a prompt change significantly increases faithfulness (from 0.70 to 0.93), it imposes heavy grounding pressure on the model, forcing it to suppress ungrounded claims and rely entirely on the provided context. If the retrieval corpus is stale, out of date, or contains inaccuracies, a highly faithful generator will accurately transcribe those corpus errors rather than relying on its parametric knowledge to produce up-to-date facts. In this scenario, the prompt change flatters a system whose remaining errors are now fully inherited from the context.

A second contributing factor is that strict grounding pressure transforms partial or uncertain answers into outright abstentions or refusals. While precision-shaped evaluation metrics treat abstentions favorably by rewarding the absence of unsupported claims, end users experience reduced coverage and missing answers, leading to higher complaint rates.

To diagnose and separate these causes, you should measure staleness on entailed claims across the corpus and track the total number of true claims generated per question while explicitly counting abstentions.