← All questions
System DesignVery Hard· Generative Retrieval· RAG· Latency

A staff engineer proposes deleting the vector index ent...

While constant-time lookup is attractive, the churn arithmetic makes this approach unsustainable. Retraining is not an upsert; it is a full training run. Furthermore, the proposal ignores that recitation is slower than prefill and that internal documents require attributable citations, which a recited passage cannot provide. The hybrid approach—keeping the index and using recitation only as a zero-hit fallback—is superior unless the corpus is static and requires no citations.

Code playground (Python)