How do you pick the learning rate for a new 70B model?

There are two primary paths. The first is standard parametrization: fit a power law on the optimal learning rate versus width from a small-scale pilot, extrapolate to the target width using the relationship η* ∝ 1/d, and validate with a short 1B-token run. The second is μP-tuning: tune the learning rate once on a 50M model and apply it directly to the 70B model, which eliminates the need for large-scale re-tuning but requires careful implementation.