What does the original 2017 transformer do differently regarding normalization, and what problem does that cause?

The original 2017 transformer utilized post-norm, which places the normalization layer after the residual addition. This design forces the gradient back-propagating to early layers to pass through LayerNorm at every single layer. Because LayerNorm rescales by the inverse standard deviation—a quantity that is highly volatile during the early stages of training—this process corrupts the effective step size at early layers, necessitating careful warmup to prevent model divergence.