How would you modify the Chinchilla analysis if you are training a mixture-of-experts (MoE) model?

In MoE models, the total parameter count differs significantly from the active parameter count. To perform a valid Chinchilla-style analysis, you must replace total parameters with active parameters in your compute formula (C ≈ 6NactiveD). You must then conduct an isoFLOP sweep specifically using active parameters to determine the correct allocation, as demonstrated by reports like Hunyuan-Large, which found ratios near 96 active-parameter tokens per active parameter.