179 questions
No questions match those filters.
← All questions
EvaluationHard· Sampling· Agreement Scores· Resolution
We sample eight times and gate on agreement. The gate w...
This is one of the questions in the full AI/ML interview bank. Pro unlocks all 1789 questions; Premium includes the same bank plus the highest daily Practice limit.
See plansThe agreement score ˆc = k/m has a standard error of sqrt(c(1-c)/m) compared to a level spacing of 1/m. Levels only separate effectively when m < 1/(c(1-c)). At m=8, the interior of the scale fails this test. Increasing m does not solve this because spacing falls as 1/m while noise only falls as 1/sqrt(m). Effective fixes include using a support classifier that evaluates the evidence or using semantic entropy over entailment clusters.