179 questions
No questions match those filters.
← All questions
System DesignVery Hard· Budgeting· Evaluation· RAG
Your eval lead wants the budget spent upgrading the jud...
This is one of the questions in the full AI/ML interview bank. Pro unlocks all 1789 questions; Premium includes the same bank plus the highest daily Practice limit.
See plansTo adjudicate, calculate the impact of each investment on the system’s error rate. Upgrading the judge model improves the verification of existing claims, but it is bounded by the quality of the underlying information. Attaching dated sources improves the oracle itself, which has a much larger impact on the actual correctness of the answers.
Unless the current judge is failing to identify obvious faithfulness errors, the budget should be prioritized toward oracle independence (e.g., dated sources) first, and grader capacity second.