← All questions
EvaluationMedium· Calibration· Long-form Generation

The model reports 80% confidence on a six-sentence answ...

A scalar confidence score on a multi-sentence answer is mathematically ill-defined without context. It could represent the probability that the entire document is correct (a point mass at 0.80) or the expected fraction of correct sentences. The interpretation depends on whether a single incorrect sentence voids the entire document or merely represents a partial error.

Code playground (Python)