← All questions
Training & Fine-tuningVery Hard· Scaling Laws· Extrapolation

By how much does a 0.03 exponent error affect a frontie...

Small inaccuracies in scaling exponents are amplified as you extrapolate further from the calibration point. At 1,000x the calibration compute (the Chinchilla scale), a 0.03 error results in a ~19% discrepancy. At 10^5x (frontier training runs), this grows to ~41%. This magnitude of error is practically significant because it can shift a recommendation from 300B to 425B parameters, which has major implications for hardware requirements and model architecture.

Code playground (Python)