Why did Kaplan underestimate the optimal token count co...

Kaplan’s research relied on multi-model comparisons using truncated training runs. Because cosine learning rate schedules require a full cooldown to yield a valid checkpoint, these truncated runs provided misleading results. Chinchilla corrected this by using the isoFLOP method, where all runs were trained to completion before comparison, effectively removing the artifacts present in the earlier study.