What is the difference between a base model and an inst...

The traditional binary distinction between base models and instruct models is no longer accurate. Modern base models are not naive text predictors; they undergo instruction-adjacent training during the final decay phase of pre-training. This process, known as WSD annealing, allows the model to learn instruction-following capabilities while minimizing the catastrophic forgetting that typically occurs during a separate, dedicated Supervised Fine-Tuning (SFT) phase.