← All questions
Retrieval & EmbeddingsVery Hard· System Design· Scalability· Trade-offs

How do you adjudicate a conflict between memory footpri...

When scaling to 500 million vectors, HNSW becomes prohibitively expensive in terms of memory (1.6 TB). IVF-PQ is much more efficient (52 GB). If the product owner mandates a specific recall level, you must ensure the chosen configuration meets it. Using IVF-PQ with an exact refine stage is a balanced approach. If this configuration exceeds latency budgets or saturates disk I/O, you must then consider architectural changes like sharding by metadata or reducing the vector dimensionality to fit the constraints.

Code playground (Python)