Design a retrieval-augmented assistant for our internal...

When designing a documentation assistant, avoid immediately jumping to vector database selections or specific chunk sizes. Instead, begin by establishing the five core design decisions and asking for three clarifying metrics: corpus size, query volume, and latency budget.

These constraints directly drive architectural choices: corpus size determines indexing and storage needs; query volume dictates throughput, caching strategy, and embedding compute; and the latency budget dictates whether to include complex rerankers, how many chunks ($k$) to pass to the model, and LLM model sizing. By grounding every choice in explicitly defined trade-offs, the architecture remains robust and defensible.