What reward signals would you use to train a reasoning...

For training a reasoning model, outcome-based rewards such as binary accuracy (correct/incorrect), format rewards (for structured tags), and language consistency rewards are preferred. Process Reward Models (PRMs) are generally avoided because they require intensive step-level human or model annotations and tend to be gamed by the model. Outcome rewards are fully self-supervised and have proven sufficient for achieving high reasoning performance.