179 questions
No questions match those filters.
← All questions
Inference & ServingHard· Kernel Fusion· Normalization· Memory Bandwidth· Optimization
A normalization layer consumes 25% of wall-clock time but less than 1% of total FLOPs. How do you speed it up?
This is one of the questions in the full AI/ML interview bank. Pro unlocks all 1789 questions; Premium includes the same bank plus the highest daily Practice limit.
See plansBecause normalization layers are memory-bandwidth-bound, techniques aimed at reducing FLOPs will not provide significant speedups. The most effective strategy is to reduce the number of parameter loads per call, for example by switching from LayerNorm to RMSNorm. Additionally, fusing kernel launches can eliminate unnecessary round-trips to HBM, significantly improving performance.