Under what conditions does μP fail to transfer the lear...

There are three primary failure modes for μP. First, learnable gains in normalization layers introduce scale-dependent implicit learning rate amplification. Second, high weight decay (typically above 0.1-0.3) disrupts the update-size invariant. Third, because the per-layer correction is derived specifically for the Adam optimizer, it does not generalize to other optimization algorithms without re-derivation.