Context windows are a million tokens now. Why not paste...

While skipping a vector index, embedding model, and retrieval stack seems appealing, it introduces prohibitive computational costs and accuracy trade-offs. The prefill processing cost scales superlinearly/quadratically with input token length. Passing a large multi-million token corpus creates thousands of times the per-query prefill compute compared to retrieving a few targeted chunks, making it cost-prohibitive at scale.

From an accuracy perspective, research demonstrates that LLMs struggle to reliably extract information buried deep within long context windows (the ‘lost in the middle’ effect), sometimes scoring below closed-book baselines. Simply expanding the context window does not guarantee the model will effectively utilize all the provided text. However, in small-scale scenarios where the documentation natively fits into a modest context window and query volume is low, direct context injection can be viable.