← All questions
Training & Fine-tuningMedium· Scaling Laws· Statistical Modeling

Did the three Chinchilla methods agree with each other?

While Methods 1 and 2 were consistent, Method 3 originally showed discrepancies because the nonlinear regression converged to a local minimum. Subsequent analysis identified non-zero-mean residuals in the Method 3 fit. By reinitializing the regression using the estimates from Methods 1 and 2, researchers were able to align Method 3 with the others, thereby strengthening confidence in the findings that D*/N* ≈ 20 and N* ∝ C^0.5.

Code playground (Python)