A staff engineer says the right fix is a nightly fine-tune, because retrieval adds a network hop and their p99 budget is 400 ms. Resolve it.

This dispute represents a trade-off between latency constraints and functional requirements. A nightly fine-tune fails the primary operational requirement: reliably generating verifiable citations to current documentation. Fine-tuning also incurs full training costs every night regardless of whether one endpoint or a hundred changed.

Conversely, retrieval adds latency via approximate nearest neighbor (ANN) lookups and potential reranking steps. To satisfy a strict 400 ms p99 latency budget, you must optimize the retrieval pipeline rather than abandon it. Strategies include using aggressive prompt/embedding caching, tuning the vector index parameters, reducing $k$ (the number of retrieved chunks), and running lightweight rerankers. The solution is to negotiate a latency budget that explicitly accommodates a lightweight retrieval stage.