179 questions
No questions match those filters.
← All questions
EvaluationHard· Safety· Helpfulness· Regression Testing
Design an eval suite that simultaneously detects regressions in safety and helpfulness after any data update.
This is one of the questions in the full AI/ML interview bank. Pro unlocks all 1789 questions; Premium includes the same bank plus the highest daily Practice limit.
See plansA robust eval suite should include a fixed red-team set of approximately 500 harmful prompts to measure safety recall and a fixed borderline set of 500 benign-but-risky prompts to measure false-positive rates. To ensure capabilities are maintained, include MMLU and a chat-style benchmark. Model releases should be gated on passing defined thresholds for all four metrics, and these should be tracked on a single dashboard to ensure that no individual regression is masked by an aggregate score.