← All questions
EvaluationMedium· Prompt Engineering· Evaluation Metrics· Sensitivity Analysis

Can you evaluate a prompt without labels?

Sensitivity and accuracy are negatively correlated, which allows a correlation to license an ordering over candidates, but the residual error prevents it from providing a precise numerical measurement. A strong strategy is to use sensitivity to shortlist candidates for GPU time and then spend the limited label budget only on the finalists. This approach acknowledges that the goal is to optimize labeling efficiency rather than avoiding labels entirely.

Code playground (Python)