179 questions
No questions match those filters.
← All questions
Retrieval & EmbeddingsMedium· Generative Retrieval· Latency· ANN
Why does generative retrieval query latency stay flat a...
This is one of the questions in the full AI/ML interview bank. Pro unlocks all 1789 questions; Premium includes the same bank plus the highest daily Practice limit.
See plansThe query cost for generative retrieval is proportional to the number of symbols needed to address the documents, which is roughly log10(N). While not strictly constant, it scales much more slowly than ANN search, which requires (1 + nprobe) * sqrt(N) comparisons. The intuition is that the model ‘memorizes’ the corpus, turning retrieval into a generation task.