179 questions
No questions match those filters.
← All questions
LLMOps & ProductionVery Hard· System Design· Latency· Evaluation
Your p99 budget is 1.2 s. The ML lead wants IRCoT with...
This is one of the questions in the full AI/ML interview bank. Pro unlocks all 1789 questions; Premium includes the same bank plus the highest daily Practice limit.
See plansA five-step loop is mathematically incompatible with a 1.2s budget, as it would lead to a significant budget violation. The design should instead gate the loop so that it only runs for complex queries. Furthermore, because learned depth predictors can degrade, the system should enforce a hard wall-clock limit, emitting a partial answer if the timeout is reached, rather than relying on a fixed step count.