We concatenated an instruction to the reranker input, r...

If the median score change under a swapped instruction is zero, the model is not utilizing the instruction tokens. This happens when all negatives are off-topic, allowing the model to satisfy the loss function using only the query. The fix is to mine ‘hard’ negatives—documents that the current reranker ranks highly but which violate the instruction—and retrain until the loss plateaus at a non-zero value.