You have a fixed per-query inference budget. How do you...

Both prefill and decode operations scale linearly with token count, allowing for a direct trade-off between context length and sampling. Rather than arbitrarily increasing top-k, which can degrade performance through position effects, one should use a diagnostic approach. Check if the gold passage is present in the retrieved context for failing queries. If absent, prioritize retrieval/reranking; if present, prioritize sampling/verification to improve the generator’s output.