Was training a 70B model on 15T tokens (over 200:1 ratio) wasteful compared to Chinchilla's 20:1 recommendation?

Chinchilla’s 20:1 ratio minimizes training loss per FLOP, whereas Llama 3’s approach minimizes total lifecycle cost (training plus serving). For models serving billions of requests, a 70B model trained on 200:1 tokens is significantly cheaper to operate than a massive 700B model trained on the same budget. Furthermore, improvements in data quality mean the loss difference between these approaches is empirically small.