Product wants agentic RAG because a competitor announce...

To resolve the conflict between Product and SRE, analyze the per-hop execution costs of agentic RAG. A single agent loop requires a tool/control routing decision, vector retrieval, and a partial generation step (e.g., ~920 ms). Running multiple multi-hop agent loops unconditionally for all traffic will systematically breach the 2-second p95 SLO.

To satisfy both teams, implement adaptive routing. Analyze query patterns to measure the actual multi-hop intent rate (e.g., 12%). Simple queries (~88%) are routed through a fast single-pass RAG path, while complex queries (~12%) execute multi-hop agentic loops. This keeps the expected average system latency within budget while providing agentic capabilities where necessary.

When implementing adaptive routing, account for routing asymmetry: a false negative (misrouting a multi-hop query to single-pass) produces an incomplete answer, whereas a false positive (misrouting a simple query to an agent loop) causes unnecessary latency overhead.