What is the training-time equivalent to inference-time...

IN2 training involves rotating positional embeddings during the training process to decouple the model’s reliance on fixed positional indices. By addressing the root cause of the position-content correlation during training, the model learns a more robust representation of sequence order. In contrast, inference-time fixes are merely wrappers that adjust attention scores after the model has already been trained, limiting their effectiveness to how the output is read rather than what the model actually understands.