← All questions
LLMOps & ProductionMedium· Retrieval Drift· Monitoring· Data Distribution

Retrieval quality drops two points a quarter. No model...

The system is self-consistent because both query and document pass through the same frozen model, but the underlying data distribution has shifted. Do not upgrade the model blindly, as this requires a costly re-index and lacks a baseline for measurement. Instead, use a frozen probe set to track performance and analyze document-frequency growth to distinguish between new tokens and new senses, which require different remediation strategies.

Code playground (Python)