179 questions
No questions match those filters.
← All questions
Inference & ServingMedium· Attention Mechanisms· KV Cache· Model Architecture
What is GQA and how does it reduce KV cache pressure?
This is one of the questions in the full AI/ML interview bank. Pro unlocks all 1789 questions; Premium includes the same bank plus the highest daily Practice limit.
See plansGQA reduces KV cache pressure by having multiple query heads share a single KV head. If there are HQ query heads and HKV KV heads where HKV < HQ, each KV head is replicated to serve multiple queries. Because the cache footprint scales with the number of KV heads, setting HKV to a fraction of HQ significantly reduces memory usage with minimal impact on model quality.