← All questions
RAGMedium· Tokenization· Prompt Engineering· Induction Heads

Few-shot accuracy dropped four points after we switched...

The drop in accuracy is likely due to the tokenizer merging the new delimiter with adjacent characters, which disrupts the pattern matching performed by induction heads. The first step is to tokenize both prompts and diff the sequences to confirm this. Moving forward, one should standardize the winning format and re-sweep whenever the tokenizer changes, as format preferences do not transfer across models.

Code playground (Python)