← All questions
Training & Fine-tuningHard· In-Context Learning· Reasoning· Evaluation

The task changes from four-way classification to multi-...

The shift to numerical reasoning breaks the local averaging theory of in-context learning, as a weighted average of labels cannot produce a derivation not present in the examples. Use a small number of demonstrations to establish the ‘show your steps’ format, but do not rely on them to cover all instances. Conduct a k-sweep: if accuracy remains flat from k=2 to k=16, it indicates the reasoning concept is not in the pre-training mixture, confirming that fine-tuning is the correct path.

Code playground (Python)