Loading today’s stories
Build1 distinct publisher3 min readUpdated
A published NVFP4 and speculative-decoding config turns a 27B open-weights model into something you can try to serve. The 206.1 tokens per second figure is single-stream and unreplicated.
The Engineer · Build desk
Compiled by The EngineerSomething wrong?How this is made
The SGLang inference project has published deployment recipes for Alibaba's Qwen3.8-27B that run the 27-billion-parameter model on a single GPU using NVFP4 quantization and DFlash2 speculative decoding [1]. The update circulated around August 20, 2026, roughly five days after Alibaba released the open weights [2], which is the gap that matters: weights are a download, a serving config is a deployment decision [3].
The model itself is a dense native multimodal model with 27 billion parameters, image and video support, a 262,144-token native context window, an extension path to one million tokens via YaRN, and an Apache 2.0 license [4]. Alibaba's model card reports 48.0152 on WildClawBench, 42.9 on Agents' Last Exam and a Pass@1 rate of 20.4 [5], and Hugging Face's evaluation metadata lists the model's WildClawBench overall rank as 8 [6]. None of that tells you anything about serving cost, and SGLang's measurements are a separate set of numbers [7].
The compression story has two moving parts. NVFP4 stores key operations in a four-bit floating-point format aimed at NVIDIA's Blackwell generation, which cuts memory and compute at the cost of precision you then have to test for [8]. DFlash2 runs a smaller draft model that proposes tokens ahead of the 27B target, which verifies them; accepted proposals let the target emit several tokens per verification pass [9]. Gains depend on prompt, draft quality, hardware, batch size and memory configuration [10]. The recipe pulls a separate draft checkpoint, incoai/Qwen3.8-27B-DFlash2, and sets --speculative-num-draft-tokens 8 [11]. That is a second checkpoint and a second compatibility surface bolted onto an already hardware-specific stack [12].
Read the benchmark conditions before you read the headline number. SGLang says it validated the RTX 5090 and RTX PRO 6000 configurations with an 8,192-token input, a 1,024-token output and concurrency of one [13]. It reports 206.1 tokens per second on one RTX 5090 [15]. At concurrency one, that is roughly five seconds of decode for the 1,024-token output [17], a single-user latency figure rather than a throughput budget for a shared endpoint. SGLang has not published acceptance-rate or output-quality data for those runs [16], so the mechanism doing most of the work in the speculative path is unmeasured in public.
There is also a discrepancy worth resolving before anyone quotes the second number. The cookbook says the DGX Spark configurations booted and served under those settings while stating the project did not take DGX Spark throughput or acceptance-length measurements [14], and the same material reports 38.28 tokens per second on a DGX Spark [15]. Those two statements do not sit together [18].
The failure mode is already documented in the neighbourhood. A separate SGLang issue reported severely repetitive output from an unofficial NVFP4 conversion of Qwen3.8-27B because an FP8 scaling value for the output head was not loaded [19]. That concerns a different checkpoint from the RadixArk NVFP4 model in the cookbook and does not indicate a defect in the new recipe [20], but it is a clean illustration of why the exact checkpoint plus runtime pair is the unit of trust in mixed-precision serving.
Context on who is doing this work: SGLang is associated with Ying Sheng, whose January 2024 overview paired a structured language for model programs with RadixAttention for reusing cached prompt prefixes [21]. Sheng later co-founded RadixArk around SGLang with Banghua Zhu [22]. RadixArk reportedly launched with a $100M seed at a $400M post-money valuation, reported by TechCrunch, with Accel leading and Spark Capital plus angels including Intel CEO Lip-Bu Tan and xAI co-founder Igor Babuschkin participating [23].
What to watch: whether anyone outside the project reproduces 206.1 tokens per second on a 5090 with the cookbook's exact checkpoints, whether SGLang publishes acceptance rates and output-quality checks for the DFlash2 path, and whether the DGX Spark figure survives clarification. Until then, treat the number as a target to verify on your own hardware, not an input to a purchase order.
Follow any of these and your For You feed starts watching them — no settings page required.
Ranked by verification strength, evidence, and original report placement.
SGLang reports 206.1 tokens per second on one RTX 5090 and 38.28 tokens per second on a DGX Spark; these are project-published measurements, not independently replicated benchmarks.
SGLang added deployment recipes for Alibaba's Qwen3.8-27B, documenting how to run the 27-billion-parameter model with NVFP4 quantization and DFlash2 speculative decoding on a single GPU.
The cookbook update and Qwen announcement circulated around August 20, 2026, about five days after Alibaba released the model's open weights, extending the launch from downloadable parameters into configurations developers can attempt to reproduce on Blackwell hardware.
Model weights alone rarely amount to a usable deployment; Sheng and SGLang's contributors are handling the less visible serving layer of the release cycle.
Alibaba's model card describes Qwen3.8-27B as a dense native multimodal model with 27 billion parameters, image and video support, a 262,144-token native context window, an extension path to 1 million tokens using YaRN, and Apache 2.0 licensed weights.
The Qwen model card reports a 48.0152 score on WildClawBench, an Agents' Last Exam score of 42.9 and a Pass@1 rate of 20.4; the WildClawBench figure does not by itself establish a ranking against other hosted systems.
Evidence-backed comparisons of source perspectives and observed adoption signals. Read the methodology
Which Builder, Operator, and Investor concerns the observed source mix emphasized—not a truth score.
Evidence, demonstrated adoption, hype gap, incentives, and confidence are assessed independently, each on its own current evidence. How these are measured.
Primary artifacts cited, no independent replication
The account rests on checkable primary artifacts — the SGLang cookbook page and its benchmark table, Alibaba's model card, Hugging Face evaluation metadata, an SGLang issue, and RadixArk's financing release as relayed by TechCrunch — which lifts it above pure announcement reporting. It is capped by a single publisher, entirely project-published performance numbers with no acceptance-rate or quality data, and an unresolved internal contradiction about DGX Spark measurement.
Documented configs, no external deployment evidence
Adoption evidence is limited to artifacts produced by the model lab and the serving project themselves: an open-weights release, a cookbook recipe, and the project's own benchmark table. No third-party deployment, usage disclosure, or reproduction by another team appears in the supplied material, and the recipe is scoped to specific Blackwell-generation hardware, so real-world uptake is unestablished.
Mildly overstated by the headline number
The headline throughput figure carries more implied weight than its basis supports: it is one project's single-stream measurement at concurrency of one, with no acceptance-rate or quality validation, and its DGX Spark companion figure contradicts the stated absence of a DGX Spark measurement. The gap stays small because the coverage itself repeatedly labels the numbers project-published and unreplicated and declines to convert the model-card scores into a ranking.
Commercial serving vendor plus cloud-demand interests
The performance numbers originate from a project whose principals run RadixArk, a venture-funded company built around SGLang, and the coverage itself frames each cookbook entry as a distribution position between open-weights labs and deploying engineers. Alibaba has a parallel interest in wider Qwen deployment supporting paid compute demand, and the cookbook's NVFP4 checkpoint is a RadixArk artifact. These incentives are disclosed rather than hidden, but they bear directly on self-reported benchmarks.
Moderate-low: single publisher, self-reported numbers
Facts about what was published — the recipe, the flags, the model card specs, the license, the financing participants — are specific and internally consistent, supporting moderate confidence. Confidence is held down by one publisher with no corroboration, performance data sourced only from the interested project, no acceptance-rate or output-quality evidence, and a direct contradiction in the DGX Spark reporting.
build
A 30B model with 3B active arrives on JumpStart, aimed at the cheap middle of agent work1 distinct publisher
product
A 27B laptop model scores like a rented one, and thinks three times as hard to do it1 distinct publisher
build
Inco AI's DFlash 2: 21% longer accepted drafts for 1.3% latency and 18.5M parameters1 distinct publisher
build
Qwen3.8's 27B dense checkpoint is the one operators can actually host1 distinct publisher
Distinct publishers with included, body-backed reporting in this cluster.
1 article · August 21, 2026