How do you resolve a conflict between a latency budget...

A nightly fine-tune is an expensive, blunt instrument that pays for a full training run regardless of the number of changes and fails to guarantee the citation of current documentation. Conversely, an ANN lookup plus reranking is a minor latency cost that can be mitigated with caching and smaller k-values. The goal is to negotiate a latency budget that accommodates a retrieval-based solution.