Design retrieval for a 100 million-document corpus serv...

When designing a large-scale retrieval system, the first step is to perform compression arithmetic based on the provided constraints. For a 100 million-document corpus, you must calculate the uncompressed size (e.g., 3,200 bytes per document) against the compressed footprint (e.g., 100 bytes using IVF-PQ). This 32x reduction is critical for determining how many shards are required to fit the index into memory. Once the capacity-bound sharding strategy is established, the next step is to identify and mitigate the recall loss inherent in the chosen compression method.