Loading today’s stories
Build1 distinct publisher3 min readUpdated
The update adds a path selector and a two-tap convolution rather than layers, recovering most of the accuracy that tripling the drafter bought at 15.2% latency, by the vendor's own numbers.
The Engineer · Build desk
Compiled by The EngineerSomething wrong?How this is made
Inco AI released DFlash 2 on August 18, an update to its parallel speculative-decoding method, and reports a 21% increase in average acceptance length for a 1.3% increase in draft-verify cycle latency [1][2][3]. The number worth arguing about is not the ratio but where it comes from: 18.5 million parameters of scaffolding bolted onto an existing five-layer drafter, rather than a bigger model [1][12][16].
Speculative decoding has a small model propose several future tokens, then has the expensive target model check them together; accepted guesses save forward passes, rejected ones are discarded, and the target model's output is preserved [9]. DFlash's contribution, in a paper submitted to arXiv in February by UC San Diego-affiliated authors Zhijian Liu, Jian Chen and Yesheng Liang, was to drop the sequential drafting loop and propose every position in a single parallel pass [6][7][8]. That trade has a known failure mode: tokens that are individually plausible can read badly when placed next to one another [10].
DFlash 2 answers with two cheap parts. A path selector keeps the top 16 candidates at each position, scores neighboring token pairs and walks those precomputed scores to pick a coherent sequence, at a cost of 2 million parameters and 0.6% cycle latency [11][12]. In Inco AI's Qwen3-4B test it produced longer accepted drafts than a DSpark correction module while using roughly 40 times fewer parameters [13]. The second part targets what Inco AI calls suffix decay, the drafter's falling accuracy near the end of a proposed block [14]. A two-tap local convolution around each attention and feed-forward sublayer lets every position mix with its immediate predecessor while keeping the computation parallel, for 16.5 million parameters, about 3% of the drafter, and 0.7% cycle latency [15][16].
The instructive comparison is the alternative Inco AI measured. Tripling the drafter from five layers to 15 recovered much the same end-of-block accuracy at a 15.2% latency cost, roughly 20 times the convolution's [17][3]. Similar accuracy, very different bill. Gains across the benchmark suite ranged from 16% to 25%, with the published Qwen figure from a five-layer Qwen3-4B drafter on GSM8K [18][4]. Output does not change, because the target model still verifies every proposed token [5].
These are the vendor's own numbers, and Inco AI concedes that headline throughput depends on model, task, sampling settings, hardware and concurrency [19]. It reports 2.7x to 3.4x autoregressive throughput for a Qwen3.8-27B drafter in SGLang at batch size one, and 3.1x to 4.6x for its Muse Glimmer drafter [20][21]. Independent results exist for the first DFlash, not DFlash 2: NVIDIA reported up to 15x higher throughput for gpt-oss-120b on an eight-GPU Blackwell system at the same interactivity target, and Google reported a 3.13x average speedup on TPU v5p in a standalone JAX benchmark plus a 2.29x end-to-end gain in a vLLM TPU pipeline comparison [24][25][26]. Those validate the original architecture across accelerators [27].
For anyone running a fleet, distribution matters more than the acceptance curve. The original repository is supported in SGLang, vLLM, TensorRT-LLM and llama.cpp, and the DFlash 2 release ships configurations for SGLang, vLLM, llama.cpp and an oMLX build for Apple Silicon [22][23]. Inco AI positions DFlash 2 as the first piece of an end-to-end inference stack for agent workloads, where a single task may run for hours and generate far more tokens than a chat session [28][29]. Agent traffic multiplies token demand and serving cost, and the pitch is fewer target-model passes without a change in output [30].
What to watch: acceptance-length measurements from someone other than Inco AI, at batch sizes above one, since the throughput figures are reported at batch size one [20][19]. Also whether the 16% to 25% spread narrows or widens on non-GSM8K work [18][4], and whether third-party accelerator tests move from the original DFlash to DFlash 2 [24][27].
Follow any of these and your For You feed starts watching them — no settings page required.
Ranked by verification strength, evidence, and original report placement.
Inco AI released DFlash 2, an updated version of the parallel speculative-decoding method DFlash, on August 18.
Inco AI reports a 21% increase in average acceptance length over the original DFlash.
The path selector and convolution responsible for the reported gain added 1.3% to draft-verify cycle latency in Inco AI's tests.
Inco AI's published Qwen benchmark uses a five-layer Qwen3-4B DFlash model on GSM8K.
Inco AI says the final output remains unchanged because the larger target model still verifies every proposed token.
The DFlash paper names Zhijian Liu alongside UC San Diego-affiliated authors Jian Chen and Yesheng Liang.
Evidence-backed comparisons of source perspectives and observed adoption signals. Read the methodology
Which Builder, Operator, and Investor concerns the observed source mix emphasized—not a truth score.
Evidence, demonstrated adoption, hype gap, incentives, and confidence are assessed independently, each on its own current evidence. How these are measured.
Detailed but single-source and vendor-measured
The technical account is unusually specific - per-component parameter counts, per-component latency shares, named benchmark configuration, and an explicit ablation against tripling drafter depth - which makes the claims checkable in principle. But every DFlash 2 number originates with Inco AI, the cluster carries one publisher, and the third-party accelerator benchmarks cover the original DFlash rather than the update. The article itself states the vendor results do not independently confirm the 21% claim.
Method widely integrated; the update itself is day-one
The underlying DFlash method has real distribution: support in SGLang, vLLM, TensorRT-LLM and llama.cpp, a CoreWeave endpoint using it by default, compatible drafters attributed to several large vendors, and 3.5 million reported Hugging Face downloads. DFlash 2 itself ships with four engine configurations plus an Apple Silicon build but has no disclosed production deployment yet, so adoption depth belongs mostly to the prior version.
Mildly overstated, and the article says so itself
The framing of a 21% acceptance-length gain for 1.3% latency and multi-x throughput leans on vendor measurement, batch-size-one conditions and a single benchmark family, while the transferable claim - lower serving cost in production agent workloads - is explicitly conditional. The gap is small rather than large because the reporting attributes the numbers, discloses the configuration dependencies, and states that the third-party results do not confirm DFlash 2.
Vendor-published numbers ahead of a commercial product
Inco AI is the primary source, is building its first public product on this method, and positions DFlash 2 as the entry point to an end-to-end inference stack - a direct interest in favourable acceptance-length and throughput figures. The corroborating benchmarks come from accelerator vendors whose own platforms benefit from demonstrating high inference throughput, and the disclosure of partner drafters and download counts is also vendor-relayed.
Mechanism claims solid, performance claims unverified
Confidence is moderate: the design and cost breakdown are internally consistent and specific enough to be falsifiable, and the ecosystem-integration facts are concrete. It is capped by a one-publisher cluster, exclusively vendor measurement for DFlash 2, and the absence of any independent or production data on whether the acceptance-length gain holds outside batch-size-one benchmarks.
build
Meta's real announcement is the split: 30B on your GPU, everything else behind the API6 distinct publishers
build
Dual 3090s, no NVLink: the serving stack broke long before the model did1 distinct publisher
build
1.5% of Hugging Face repos take 99.2% of downloads, and the ceiling is Chinese1 distinct publisher
science
The real disclosure in Qwen3.8-Max is the rack: 2.4T open weights, 72 GPUs, 4K tokens/sec1 distinct publisher
Distinct publishers with included, body-backed reporting in this cluster.
1 article · August 19, 2026