← All questions
RAGMedium· Training Objectives· Invariance· Supervised Learning

Why is the target identical for every retrieved chunk i...

The constant target asserts that the label y belongs to the question, not the specific context-question pair. By training on all instances, we force the model to learn to extract information only when the context is useful. The non-supporting instances are valuable because they force the model to rely on its parametric memory and learn to ignore noise, rather than simply memorizing a specific chunk as the ‘correct’ answer.

Code playground (Python)