We’re fine-tuning with IN2-style rotated-position super...

Rotated-position supervision is effective at teaching the model that useful information can exist far from the point of generation, which helps close the recency-side gap. However, it does not address the primacy bias.

If the chat template remains static, with the system message fixed at position 0 throughout the fine-tuning process, the model will continue to treat position 0 as uniquely high-leverage. To fix the entire U-curve, you must validate performance using a position-controlled probe and introduce interventions, such as varying the system message’s position or salience, to specifically target the primacy bias.