DeepSeek-V3 claims 'auxiliary-loss-free balancing' but...

The online bias (bi) replaces the batch-level auxiliary loss by reactively adjusting expert attractiveness without adding a differentiable loss term to the training objective. Because bi relies on batch-level running averages, it is effective during training but cannot prevent a single unusual inference sequence from overloading specific devices. The sequence-wise auxiliary loss acts as a separate, lighter regularizer that operates within individual sequences to ensure routing robustness at inference time, addressing fast spikes that the training-time bias cannot.