← All questions
Retrieval & EmbeddingsHard· Retrieval· Model Capacity· Label Quality

How do you diagnose and address flat retrieval quality...

When retrieval quality remains flat across capacity, it indicates a ceiling in the labels. You should diagnose this by sampling the teacher ‘s’ times on an audit slice and counting how many errors survive all draws to estimate ϵsys. Once the ceiling is confirmed, propose an intervention in the labeling process. If asked why not simply buy more labels from the same teacher, the answer should be based on the correlation coefficient (ρ) between different teacher families.

Code playground (Python)