What limits expert granularity, and where does DeepSeek...

Two primary factors limit expert granularity. First, tensor core efficiency: matrix multiplication performance drops if the inner dimension (dexpert) falls below the GEMM tile size, which is roughly 512 for H100s in bfloat16. At m=4, dexpert is 1024, which is well above this floor. Second, expert parallelism communication: increasing the number of experts leads to more all-to-all calls with smaller payloads, which can degrade performance if the communication-to-compute ratio becomes too high. At m=4, this ratio remains favorable for high-speed interconnects like InfiniBand or NVLink.