If you eliminate token dropping at inference time by ro...

When a model is trained under a regime where overflow tokens are dropped, it calibrates its downstream layers based on those specific activation statistics. If you introduce re-routing at inference time, you change the effective per-expert load distribution. For mildly imbalanced batches, this discrepancy is negligible. However, for heavily skewed batches—such as a batch of code-heavy prompts on a general-purpose model—the shift can be meaningful. A common mitigation is to include no-drop re-routing during fine-tuning so the model learns to handle both modes.