← All questions
Inference & ServingHard· Cascade· Inference Efficiency· Model Architecture

UAR’s figure shows four classifiers in parallel and a d...

Training is parallel, utilizing four independent binary heads over the same frozen base model hidden state without shared loss. During inference, the system functions as a cascade. However, because all four probes read a hidden state computed once, the cascade does not save computation time unless one of the criteria requires a generation step rather than a simple matrix multiply. Therefore, it is most efficient to evaluate all four and apply the decision tree to the resulting bits.

Code playground (Python)