How does DeepSeek-V3 achieve load balance without an au...

DeepSeek-V3 achieves load balance by adding per-expert bias scalars (bi) to routing scores during top-K selection. These biases are excluded from the gate weights. An online controller adjusts these biases by ±γ each step based on the load of the expert in the previous batch. This process is gradient-free, as the optimizer only sees standard language modeling gradients.