Loading today’s stories
Build1 distinct publisher3 min readPublished
Mercury reports over 1,000 tokens per second per user, and Nemotron Diffusion claims 2 to 8 times autoregressive throughput. Both are vendor figures. The sampler that produces them charges for its speed in arithmetic.
The Engineer · Build desk
Compiled by The EngineerSomething wrong?How this is made
Count the forward passes. Autoregressive decoding of N tokens costs N sequential passes, because each step conditions on the token the previous step emitted, which is why you cannot parallelise your way out of it [6]. The masked diffusion loop has a different shape: it runs a fixed number of steps, and each step calls the model on the whole sequence, blanks and settled positions alike, then re-masks most of what it just guessed while keeping slightly more tokens fixed than the round before [5]. Run S steps over a length-N sequence and you have evaluated S times N token positions where the autoregressive decoder evaluated N [8]. You spend arithmetic to buy fewer round trips.
That bounds where the reported speed can come from. Inception Labs' per-user figure implies a budget under one millisecond per emitted token [1][9]. For that to transfer, your GPU has to have arithmetic left over at the batch size and sequence length you actually serve, and your step count has to stay well below your output length. Long outputs help the trade. Large concurrent batches, which already saturate the machine, do not.
The quality figure needs the same reading. Both numbers NVIDIA reports are range tops: the throughput multiple and the retained quality [2]. "Up to 99 percent" constrains the best case and says nothing about the worst, and the write-up relaying it does not name the tasks or the eval suite. Its author discloses that he is a backend engineer who has never trained a diffusion model and is working from this week's primary sources, including the Kuleshov group's build guide and Sander Dieleman's post on continuous diffusion [13][4]. That is honest sourcing, and it means these are vendor claims about someone else's workload until somebody publishes the harness.
The mechanism itself is good craft. Masked diffusion is close to a generative BERT, and the detail that makes it generative rather than a fill-in-the-blanks classifier is that the masking rate is randomised across training instead of fixed [11]. That one training-time choice, plus a principled objective, is credited with closing most, though not all, of the quality gap with autoregressive models [12]. The sampling loop is about ten lines of pseudocode [14], which is a low bar for a research idea to clear and a rare one to actually clear.
So left-to-right is worth pricing again rather than assuming. Speculative decoding and KV caching shave autoregressive cost, but they leave the sequential loop untouched [7], while diffusion changes that loop and hands you the bill in FLOPs instead. In my context, short structured outputs on a box with idle arithmetic between dependent steps, that is the right trade. If your fleet is already compute bound at peak, it is the wrong one, and no step schedule fixes that.
Ranked by verification strength, evidence, and original report placement.
Autoregressive decoding has no error correction: once a token is emitted it is permanent, early mistakes compound because later tokens condition on the flawed ones, and causal attention means the model never looks at future context.
Google shipped Gemma Diffusion as an open-weights release.
Two explanatory posts circulated on Hacker News in the same week: a guide from the Kuleshov group at Cornell titled "How to Build a Diffusion Language Model", and Sander Dieleman's post on continuous diffusion language models.
Masked diffusion generation starts from a sequence of all blanks, asks the model to fill in every blank, accepts the full guess, then re-masks most of the sequence while keeping slightly more tokens fixed than the previous round, and repeats; the text assembles out of order, wherever the model is most confident.
Autoregressive generation of N tokens takes N sequential forward passes, and cannot be parallelised because each step depends on the previous one.
Speculative decoding and KV caching shave the cost of autoregressive decoding, but the fundamental loop stays sequential.
Distinct publishers with included, body-backed reporting in this cluster.
dev.to
1 article · August 30, 2026
Follow any of these and your For You feed starts watching them — no settings page required.
security
Washington names industrial-scale distillation, then hands the detection bill to abuse teams1 distinct publisher
product
Washington's secret AI test is coming for open weights, and release dates go with it2 distinct publishers
product
Swapping out the GPU leaves four more rack lines on Nvidia's invoice1 distinct publisher
invest
Nvidia's August 26 print: 92% of the quarter rides on one segment1 distinct publisher
Evidence-backed comparisons of source perspectives and observed adoption signals. Read the methodology
Which Builder, Operator, and Investor concerns the observed source mix emphasized—not a truth score.
Evidence, demonstrated adoption, hype gap, incentives, and confidence are assessed independently, each on its own current evidence. How these are measured.
Mechanism checkable, numbers borrowed
The story splits cleanly in two. The mechanism half you can audit from your chair: dev.to prints the sampling loop, and it traces the masked diffusion formulation and its four fixes back to the Cornell guide it read. The numbers half is entirely borrowed — Mercury's thousand tokens a second, Nemotron's 2-to-8x, the 99 percent quality retention, and the assertion that a principled objective 'closed most of the quality gap' all arrive without a benchmark, a baseline model, or a hardware line.
Models shipped, usage uncounted
Three named models from three organisations, two with weights out the door, is real shipping rather than a preprint cluster — and the Hacker News explainer traffic says builders are actually reading. What no one in this reporting counts is a deployment, a customer, a token volume, or a price. Diffusion language models exist and are downloadable; whether anything runs on them in production is simply not in evidence here.
Speed priced in tokens, not in passes
Credit where it is due: this reporting flags its own headline numbers as vendor figures and lists what plain masked diffusion still gets wrong. The overstatement is structural rather than rhetorical. Speed is quoted in tokens per second while the sampler is billed in whole-sequence passes, and the piece never puts those two units in the same sentence — so 'two to eight times faster' reads as free when the mechanism it just explained charges S full-sequence evaluations to emit N tokens.
The sellers supplied the benchmarks
Inception Labs sells Mercury; NVIDIA sells the accelerators these throughput ratios are measured on and now ships the models too; Google is competing for the same open-weights mindshare. Every number that makes diffusion look fast came from a party with something to gain from it, and nothing in this reporting is adversarial to those numbers. The author's own incentive is the mild one — a well-read explainer on a developer platform — and he discloses it up front.
One explainer, no second read
We are reading a single dev.to piece published the week two diffusion explainers trended, and it is the only account of this story we have. That is enough to be confident about how masked diffusion sampling works, because the algorithm is short enough to check. It is not enough to be confident about any figure, ratio, or quality claim in it — those would need a measurement someone other than the vendors performed.