Router accuracy is 82% on the held-out set, but end-to-...

Relying on aggregate scalar accuracy masks the severity of specific error types. Routing a knowledge-intensive query to ‘answer directly’ removes necessary evidence, leading to fluent hallucinations, whereas routing a simple lookup to a more complex retrieval path only adds latency. The fix involves using a class-weighted loss or an asymmetric margin threshold that biases the system toward retrieval, and measuring quality improvements on a per-route basis rather than in aggregate.