← All questions
RAGHard· Classification· Automated Labeling· Model Routing

Where does the training data for a complexity classifier come from, given that nobody labels hop counts?

The training data is derived by running your available pipelines on existing QA pairs and scoring them using exact match. You keep the cheapest configuration that yielded a correct answer and fall back to the source corpus bias for queries where no pipeline succeeded. This approach ensures the label is a property of your model’s memory rather than a heuristic based on query length or entity count. Note that because this label depends on the generator, it must be re-derived whenever the model is swapped.

Code playground (Python)