179 questions
No questions match those filters.
← All questions
LLM FoundationsMedium· Transformer Architecture· Attention· MLP
Walk me through a decoder-only transformer block. What...
This is one of the questions in the full AI/ML interview bank. Pro unlocks all 1789 questions; Premium includes the same bank plus the highest daily Practice limit.
See plansEach decoder-only transformer block contains a self-attention sub-layer followed by an MLP sub-layer. These are wrapped in pre-norm residuals, where normalization occurs before the sub-layer to maintain signal integrity across deep stacks. The MLP typically uses SwiGLU activation. The input is added back to the output of each sub-layer, maintaining the residual stream throughout the depth of the model.