Build1 distinct publisher3 min readPublished
A new paper fits log-linear laws for draft acceptance rate against pretraining tokens, draft capacity and decoding batch size, and argues with a roofline model that the tree you tune at batch 1 is the wrong tree in production.
The Engineer · Build desk

Compiled by The EngineerSomething wrong?How this is made
A draft model earns its keep out of slack. Token-by-token decoding is memory-bandwidth bound, so arithmetic units idle while weights stream, and speculative methods such as Medusa and the EAGLE series spend that idle window verifying several predicted tokens at close to the cost of one step [7]. That is the entire trick. It also tells you why batch size belongs in a scaling law at all: batching is the other consumer of the same slack. Group enough sequences into one decode step and the verifier's spare capacity is already sold to other users' tokens. Most published work sits at batch 1, while industrial serving decouples prefill from decode and runs much larger batches [8].
The two supply-side laws are the ones you can budget against. Prior LLM scaling work related cross-entropy loss to model size, dataset size and FLOPs by power law [9]; here the dependent variable is draft acceptance rate and the reported shape is log-linear in pretraining tokens and in draft capacity [1]. Take that form literally. If acceptance is a*log2(T) + b, then doubling the draft's pretraining tokens adds a, and doubling again adds another a [1]. The first few doublings are cheap and the tenth costs the same in acceptance as the second while costing 256 times the data. A log-linear law is a promise about where you stop, not just about where you gain.
Now the arithmetic on the headline figures. Read the acceptance range against EAGLE2 and the engine-level throughput number as multipliers, and the throughput figure lands above the midpoint of the acceptance range, 1.85 [2]. That is roughly what the mechanism predicts, since throughput also has to pay for draft forward passes and tree overhead, and the two measurements were not taken under the same conditions anyway.
For the throughput number to transfer, several things have to hold in your stack. The backbone has to be dense, since the study is conducted on dense architectures and the systems are built on Llama2/3 and Qwen2.5 [10][2]. Your traffic has to resemble the workloads where the paper puts its peak gains, summarization and QA, which by definition bounds everything else lower [5]. Your baseline has to be EAGLE2, because against EAGLE3 the reported acceptance margin is 0.3 [3]. And your serving engine has to run the batch regime the third law was fit in [8]. An arxiv HTML render that eats the multiplication signs and thresholds is a poor advertisement for a paper about numbers, so treat the exact figures as provisional until the PDF or the code says otherwise.
The framing deserves credit. Draft-model research has mostly gone at architecture and sampling strategy and left the pretraining and SFT scaling question alone [12], and fitting curves on tokens, capacity and batch turns draft sizing into interpolation instead of a sweep you rerun every time the backbone changes. The paper also reports Scylla using only the 8x SFT data augmentation from EAGLE, which reads as a claim that the gains came from scale rather than data tricks, though that sentence is truncated in the HTML we read [13]. Code is promised later [11]. The number I would want before committing a draft-model budget is the acceptance curve refit on my own backbone at my own batch size, and that needs the release.
Ranked by verification strength, evidence, and original report placement.
Building on the laws, the authors present Scylla, which coordinates multi-dimensional scaling for popular LLMs (Llama2/3, Qwen2.5).
The paper states that Scylla uses only the 8x SFT data augmentation from EAGLE; the sentence is truncated in the HTML version.
A paper titled "Scaling Laws for Speculative Decoding" reports log-linear scaling laws (Theorems 1.1, 1.2 and 1.3) governing draft model acceptance rate, or decoding speed, across three dimensions: pretraining token volume, draft model capacity, and decoding batch size.
Speculative decoding methods including Medusa and the EAGLE series predict multi-step tokens with minimal overhead and verify them with underutilized computational resources; memory-bandwidth constraints underutilize compute during token generation.
Existing speculative decoding studies typically focus on single-batch decoding, whereas industrial deployments often decouple prefill and decoding phases and process larger batches for efficiency.
Prior LLM scaling law studies revealed power-law relationships between cross-entropy loss and computational factors including model size, dataset size and FLOPs.
Distinct publishers with included, body-backed reporting in this cluster.
1 article · August 27, 2026
Follow any of these and your For You feed starts watching them — no settings page required.
build
SPEED-Bench re-tests speculative decoding at the batch size you actually serve1 distinct publisher
build
The judge went synthetic first, which tells you which part of your pipeline is next1 distinct publisher
invest
Meta FAIR says the standard way to plan a training run costs 10x more than it needs to1 distinct publisher
invest
Tiny corp wants Etched's numbers. Jane Street led $700M at $21B without publishing any1 distinct publisher
Evidence-backed comparisons of source perspectives and observed adoption signals. Read the methodology
Which Builder, Operator, and Investor concerns the observed source mix emphasized—not a truth score.
Evidence, demonstrated adoption, hype gap, incentives, and confidence are assessed independently, each on its own current evidence. How these are measured.
Strong shape, missing numbers
The qualitative findings are clearly stated and self-consistent, and the authors say the laws held across more than ten runs. But the quantities a reader would check are absent from what arXiv actually renders: the temperature behind the EAGLE2 comparison, the batch size where throughput collapses, the multiplication signs on 1.5-2.2 and 2, and the coefficients of the theorems themselves. A law without its constants is a direction, not a measurement.
Nothing anyone outside can run
Adoption at this point consists of one posting and one deployment described by the people who built both. Code is deferred with no date, the inference engine is unnamed, and no third party has reported trying Scylla. The EAGLE2 and EAGLE3 baselines it is measured against are the widely used ones - which is what makes the comparison interesting and also what makes the absence of any independent run conspicuous.
Transformative potential, unrendered denominators
The abstract reaches for 'validating the transformative potential of systematic scaling' on the strength of results whose conditions did not survive the trip to HTML. The gap is real but not enormous: the claims are specific, bounded, and internally coherent - the reported 2x engine throughput sits sensibly above the 1.85 midpoint of the acceptance-rate range - and the authors are candid that RLHF's effect on draft accuracy is untouched and that the code is not out yet.
Proponents choosing their own baselines
Scylla's authors defined the comparison, ran the baselines, owned the serving engine and set the batch configuration that produced the best number. The care taken to note that Scylla borrows only EAGLE3's 8x SFT augmentation, minus hidden fusion and train-time test features, is an argument for credit as much as a methods note. None of that makes the results wrong; it does mean every favourable choice in the setup was available to the same party, and the code that would let others re-run it is still pending.
Confident about what is claimed, not about what is true
We can say with near certainty what this paper argues and how it argues it - the text is unambiguous on the three laws, the recipe and the batch-size warning. What we cannot do is rate the numbers, because one unreviewed source, a set of unrendered constants and a pending code drop leave no way to separate a real result from a well-configured one.