What does the LR landscape look like during a μP width...

A key advantage of μP is that the optimal base learning rate remains consistent across different model widths (e.g., 128, 512, and 2048). In contrast, under standard parameterization, the optimal learning rate valley shifts to the left as width increases, meaning that a learning rate found to be optimal for a small proxy model will likely be suboptimal for a larger production model.