Why isn't a cross-encoder used as a retriever despite i...

A weak answer simply states that cross-encoders are too slow. A strong answer explains the structural property: the bi-encoder’s score factorizes into sim(η(q), η(d)), allowing η(d) to be computed at index time. In contrast, the cross-encoder’s score is a function of the pair and cannot exist before the query is known. Quantitatively, this means the cross-encoder requires N joint forward passes per query, making even smaller cross-encoders unusable for large-scale retrieval.