← All questions
EvaluationEasy· Metrics· Instruction Following· NLP

Why can’t you use exact match or BLEU for instruction-f...

Instruction following is inherently open-ended; a single prompt can have thousands of valid, distinct responses, meaning exact match will almost always return zero. BLEU is similarly inappropriate because it penalizes paraphrasing, which is a core feature of helpful chat models. Evaluation must instead rely on verifiable structural constraints or semantic judges.

Code playground (Python)