Chinchilla Scaling Law

Meta FAIR says the standard way to plan a training run costs 10x more than it needs to

A new paper argues Chinchilla treats model size and data as independent when they are not, and that fixing it lets teams fit the same scaling curve on a sparse grid.