179 questions
No questions match those filters.
← All questions
Inference & ServingMedium· KV Cache· GQA· MQA
What is the difference between MQA and GQA-8, and when...
This is one of the questions in the full AI/ML interview bank. Pro unlocks all 1789 questions; Premium includes the same bank plus the highest daily Practice limit.
See plansMulti-Query Attention (MQA) uses a single KV head for all query heads, offering the maximum possible reduction in KV cache. GQA-8 uses a moderate number of KV heads. Because MQA can lead to measurable quality degradation in large models, GQA-8 is preferred as the standard. MQA is only recommended when the memory budget is so tight that even the reduced footprint of GQA-8 is insufficient.