179 questions
No questions match those filters.
← All questions
Training & Fine-tuningHard· Attention Mechanisms· Model Design
You’re training a new 70B model from scratch. A colleag...
This is one of the questions in the full AI/ML interview bank. Pro unlocks all 1789 questions; Premium includes the same bank plus the highest daily Practice limit.
See plansGQA is a deliberate architectural choice made at initialization to optimize inference economics. To resolve the debate, train two identical 7B models—one with MHA and one with GQA (e.g., G=8). Evaluate them on a suite of downstream tasks and measure the KV cache size at the target serving batch size. If GQA shows less than 1% degradation, the significant inference cost savings justify its adoption for the 70B run.