← All questions
Inference & ServingEasy· Latency· Performance Monitoring· System Design

A dashboard shows a healthy median latency, but users r...

Aggregate metrics like the median can be misleading when multiple sequential stages contribute to the total latency. You should break out p95 and p99 latency metrics by stage to identify where the tail-end delays are occurring. This helps explain why a large population of users might experience slowness despite a ‘green’ median metric.

Code playground (Python)