How would you detect and fix length bias in a deployed RLHF model?

To detect length bias, instrument production request logs to monitor output token counts by query category and compare them against the SFT baseline. If length is inflated, address it in the next training run by implementing length-normalized objectives. As a short-term mitigation, apply a maximum-token constraint directly in the inference server.