Walk through the complete RAG pipeline, start to finish.

RAG splits cleanly into two phases that run on very different schedules. Indexing runs once, offline, as a batch job: documents get loaded, split into chunks, each chunk gets embedded by an embedding model, and the embeddings — plus the original chunk text — go into a vector database. For a corpus of 500 documents split into 15,000 chunks, that whole process might take half an hour and a few dollars in embedding costs.

Query time runs on every single user request, and its cost profile matters much more because it’s what users actually feel. The incoming question gets embedded with the same model used at indexing time, the vector database returns the top-K most similar chunks by cosine similarity, those chunks get inserted into the LLM’s prompt as context, and the model generates an answer grounded in that retrieved text — a couple of seconds and a fraction of a cent per query, versus the much larger one-time indexing cost. Keeping the two phases mentally separate is useful because they get optimized differently: indexing for throughput and coverage, querying for latency and precision.