How do you monitor an LLM system in production?

Across four layers, each watched independently rather than folded into one dashboard:

• Inputs — drift in the distribution of what’s actually arriving. This is the earliest warning signal and the one most commonly left unmonitored, because it requires watching the traffic itself, not just the system’s output.
• Model behavior — output distribution, refusal rate, schema-validation failure rate, and latency and cost per stage.
• Retrieval — measured independently of final answer quality, because a strong generator can mask a quietly degrading retriever for a long time before it shows up downstream.
• Humans — acceptance rate, override rate, escalation rate, abandonment. These measure trust, which is often the leading indicator that something is wrong before any automated metric moves.

The thing people forget in regulated environments specifically: the observability pipeline itself has to respect data residency. Shipping prompts and outputs to an external observability tool can exfiltrate exactly the content the client was promised would never leave their environment.