Meta FAIR says the standard way to plan a training run costs 10x more than it needs to

Meta's FAIR lab has published a paper arguing that DeepMind's Chinchilla scaling law mis-specifies the relationship between model size and training data, and that a corrected version reaches near-identical accuracy while consuming roughly 10x less compute to fit ^[1][3][8]. The money at stake is not in the headline training run but in the grid of small profiling runs teams burn before it, which on this account has been sized about an order of magnitude larger than necessary ^[8][13].

The mechanism is unglamorous. Chinchilla, established by DeepMind in March 2022, treats parameters (N) and training tokens (D) as independent variables that each contribute separately to final loss ^[2]. The Skaling authors computed the mixed partial derivative of the loss surface with respect to N and D and report that it is non-zero ^[4]: the payoff from adding parameters depends on how much data you have, and the reverse. Their proposed form wraps the Chinchilla expression in an exponent, L(N, D) = (A / N^a + B / D^b)^k + E, where k greater than 1 encodes the coupling ^[3]. Set k to 1 and you get Chinchilla back exactly, which means the old law is a special case rather than a competitor ^[3][14].

The reported margins, according to Cryptobriefing's account of the paper, are wide but not universal: Skaling beat Chinchilla at 76% of tested configurations, with a median 2.2x improvement in prediction accuracy, and improvements of 4x or better at a third of tested points ^[5][6]. Mean absolute percentage error fell by 1.5 to 3x across scenarios ^[7]. That also leaves 24% of configurations where it did not win ^[15]. Note what is being measured: accuracy in predicting loss, not better models.

The line item is the profiling grid. Skaling is reported to hold its accuracy on a sparse "L-shaped" grid, where you vary one dimension with the other fixed and then reverse, instead of the full grid Chinchilla requires, at roughly 10x less compute ^[8]. If that survives contact with other labs, the implication for anyone budgeting model development is that about 90% of the compute historically allocated to scaling experiments was buying precision that a two-arm sweep already delivers ^[8][13].

It matters most for the way models are actually shipped. Chinchilla's 2022 contribution was to show that many models were badly undertrained ^[10], but production models are now routinely overtrained well past its optimum because inference economics reward small models fed enormous amounts of data ^[11]. The coupling exponent is presented as capturing how the optimal balance moves in exactly that regime ^[12], and at frontier compute levels the paper reports optimal token-to-parameter ratios that can differ from Chinchilla's by orders of magnitude ^[9].

What to watch: independent refits by teams with their own loss surfaces, since the reported result is a ratio and the account gives no absolute compute figures or fitted values of k ^[15]. Watch whether the 10x saving holds when the L-shaped grid is used to extrapolate to a run much larger than any point in it, which is the only use that matters. And watch the disclosed token-to-parameter ratios of models released over the next two quarters; if the coupling correction is real, those ratios should start moving away from Chinchilla-derived defaults ^[9].