179 questions
No questions match those filters.
← All questions
RAGVery Hard· Latency· Recall· Fusion-in-Decoder
Serving wants to cut k from 40 to 5 to hit a p99 target...
This is one of the questions in the full AI/ML interview bank. Pro unlocks all 1789 questions; Premium includes the same bank plus the highest daily Practice limit.
See plansRefuse the framing that k is a single knob for quality vs. latency. Analyze the rate at which gold passages appear in the top 5 vs. top 40. If the ‘cliff’ is explained by this rate, it is a recall problem solvable by a reranker. If recall is flat but quality drops, the fusion is genuine. If fusion is real, the trade-off should be keeping k=40 and reducing z, as total compute cost is m=kz.