DCLM-baseline filters 240 trillion tokens down to 3.8 t...

Aggressive quality filtering can become a capacity constraint that limits the scale of training runs before compute itself becomes the bottleneck. A senior engineer recognizes that filtering 240 trillion tokens down to 3.8 trillion may be overly restrictive. Alternative approaches, such as using a softer classifier ensemble (like the Nemotron-CC method), can help recover more tokens while maintaining acceptable quality, highlighting the constant tension between data coverage and data quality in pipeline design.