A user reports that your MoE API returns different outp...

In Mixture-of-Experts (MoE) models, token dropping is often batch-dependent. If a prompt is processed alongside other high-traffic sequences, the experts it requires may be saturated by those other requests. When an expert is saturated, tokens from the prompt are dropped at that layer, which changes the internal representation of the input and results in different outputs for the same prompt. To fix this, one can implement no-drop inference routing or ensure request isolation.