179 questions
No questions match those filters.
← All questions
EvaluationHard· Metrics· Copyright· Audit
Our audit reports mean token overlap with copyrighted w...
This is one of the questions in the full AI/ML interview bank. Pro unlocks all 1789 questions; Premium includes the same bank plus the highest daily Practice limit.
See plansThe mean is an inappropriate metric because, under lock-on, the distribution is bimodal; the mean is largely determined by generations that never locked on and is independent of the tail that exhibits memorization. You should instead request the maximum run at production k over prefixes from named works. If the maximum is too noisy, use a high-percentile variant as the tracked metric, while keeping the maximum as the gate.