Your dense retriever misses queries containing product...

When a dense retriever fails on specific entities like product SKUs, it is often due to the limitations of the embedding model’s tokenizer or the loss of information during vector aggregation. Simply scaling up the embedding model will not resolve this, as the underlying representation failure persists. The correct approach is to implement a hybrid retrieval system by adding a sparse retrieval leg, which can explicitly match exact terms like SKUs that the dense model fails to capture. Proof of success is established by demonstrating improved recall on SKU-heavy queries after integrating the sparse component.