How would you track training health during a long pre-t...

During a long pre-training run, you should monitor validation perplexity on a domain-stratified held-out set, tracking both aggregate and per-domain curves. It is critical to set up automated alerts for two specific events: perplexity spikes, which indicate potential data or learning-rate issues, and validation-training divergence, which signals overfitting or data contamination. Task accuracy is generally too slow to provide the real-time feedback required during the training process.