← All questions
EvaluationHard· Reranking· Debugging· Ranking Metrics

A listwise reranker is deployed but nDCG@10 is indistin...

Start by checking if the model is actually performing any reordering; a broken model that falls back to the input order will produce the same metrics as a useless model. If the permutations are real, check the schedule. A head-first slide or a small window size can cap the promotion of documents, preventing the model from improving upon the BM25 baseline. Finally, use recall@10 conditioned on the gold starting below rank 50 to see if the model is failing on specific slices.

Code playground (Python)