179 questions
No questions match those filters.
← All questions
EvaluationMedium· Calibration· ECE· Abstention Gates
Our confidence scores have an ECE of 0.02. Are we well...
This is one of the questions in the full AI/ML interview bank. Pro unlocks all 1789 questions; Premium includes the same bank plus the highest daily Practice limit.
See plansAn ECE of 0.02 means that the model’s stated confidence matches its empirical accuracy within specific buckets. However, this metric says nothing about the model’s ability to rank correct answers higher than incorrect ones. A counterexample is a model that always emits the base rate as a constant; it would have an ECE of 0 but an AUROC of 0.5, rendering it useless for thresholding. To build an effective abstention gate, one must look at the risk-coverage curve rather than relying solely on calibration metrics.