You swapped the first-stage retriever from BM25 to a de...

Separate two hypotheses: if the reranker is order-sensitive, shuffling the candidate order under the old retriever already moves NDCG@10 by a comparable amount, whereas if the candidate distribution is what changed, shuffling is flat and the drop concentrates in queries whose negatives moved most. Order sensitivity in a set-scoring model means positional information leaked in - through an actual positional embedding, or through training on lists that always arrived in BM25 order. Retrain with the candidate order shuffled per epoch, then reintroduce the prior deliberately as an ordinal embedding if the ablation says it earns its place.