← All questions
Retrieval & EmbeddingsVery Hard· Embedding Models· Infrastructure· Efficiency

The retrieval team wants to ship a new embedding model...

The disagreement between the retrieval and platform teams is a classic efficiency trade-off. Bi-encoders rely on precomputed vectors to ensure fast retrieval, but these vectors are strictly bound to the model weights used during their creation. Consequently, any change to the embedding model invalidates the entire existing index. To adjudicate, you need to quantify the operational cost of re-embedding 500 million chunks. If the cost of re-indexing exceeds the performance gains of the new model, the monthly cadence is likely unsustainable.

Code playground (Python)