DeepSeek-V3 claims to be auxiliary loss-free but still...

The distinction is meaningful from an engineering perspective. The traditional batch-level auxiliary loss has a large coefficient and causes significant gradient interference because its direction often conflicts with the language modeling gradient. The sequence-wise loss used by DeepSeek-V3 has a 3-10x smaller coefficient, targets within-sequence imbalance at inference time, and causes far less interference because routing distributions are already partially balanced by the learned router.