179 questions
No questions match those filters.
← All questions
LLM FoundationsHard· Pre-training· Attention· Bias
If primacy comes from the system message always sitting...
This is one of the questions in the full AI/ML interview bank. Pro unlocks all 1789 questions; Premium includes the same bank plus the highest daily Practice limit.
See plansThe system-message explanation only accounts for the instruction-tuned phase of model development. Base models, however, exhibit a related bias because of their pre-training objectives.
In large-scale pre-training, documents often front-load context, making the initial text of a document disproportionately predictive of the subsequent content. This creates a pre-training-level prior for primacy. When a model is later instruction-tuned with a fixed system message, these two priors compound, reinforcing the U-shaped performance curve.