What does Supervised Fine-Tuning (SFT) teach a model, a...

Supervised Fine-Tuning (SFT) effectively teaches a model to mimic the style and content of high-quality demonstration data. While this significantly improves instruction-following and surface-level helpfulness, it does not inherently optimize for human preference between different possible responses. Because the model is trained to maximize log-likelihood on provided examples, it learns what was said rather than what is preferred. This approach fails to teach the model how to refuse harmful requests not explicitly shown in the training set and often forces the model to provide confident, yet potentially incorrect, completions in domains where it lacks factual grounding.