← All questions
Retrieval & EmbeddingsHard· HNSW· Skip Lists· Vector Search

Derive HNSW from a skip list and determine the number of layers for ten million vectors.

To derive HNSW from a skip list, you substitute the skip list’s total order with a proximity graph structure. The halting test is modified such that the search stops when no neighbor is closer to the query vector than the current node. The height of the graph is determined by the probability distribution Pr[ℓ ≥ j] = M^-j. For a dataset of ten million vectors (n = 10^7) and a branching factor of M=16, the formula L = logM(n) yields approximately six layers.

Code playground (Python)