You are evaluating a closed-weight model and the developer claims the test set was not in training data. You have no access to training data. How do you verify?

When you lack access to the training data of a closed-weight model, you must rely on behavioral probes to detect contamination. An ordering probe checks if the model’s preference for specific answer-choice orderings correlates with the original benchmark’s ordering. A confidence asymmetry test checks if the model assigns systematically higher log-probabilities to the original test examples than to semantically equivalent paraphrases created after the training cutoff. Using both tests together provides a robust signal for verifying the developer’s claims.