Faithfulness is 0.94 and climbing, and complaints are c...

When faithfulness metrics improve but user complaints increase, it suggests that the system is becoming more ‘faithful’ to incorrect or outdated information.

First, estimate the error rate by replaying known-false claims. Second, audit the gold set to ensure it hasn’t become stale. Finally, check for information-access symmetry; often, attribution judges fail because they do not have access to the same information as the human annotator, leading to false negatives. Ultimately, driving faithfulness to 1.0 may simply be transcribing the corpus’s inherent error rate.