SGLang's one-GPU Qwen3.8-27B recipe is the useful half of the release

The SGLang inference project has published deployment recipes for Alibaba's Qwen3.8-27B that run the 27-billion-parameter model on a single GPU using NVFP4 quantization and DFlash2 speculative decoding ^[2]. The update circulated around August 20, 2026, roughly five days after Alibaba released the open weights ^[3], which is the gap that matters: weights are a download, a serving config is a deployment decision ^[4].

The model itself is a dense native multimodal model with 27 billion parameters, image and video support, a 262,144-token native context window, an extension path to one million tokens via YaRN, and an Apache 2.0 license ^[5]. Alibaba's model card reports 48.0152 on WildClawBench, 42.9 on Agents' Last Exam and a Pass@1 rate of 20.4 ^[6], and Hugging Face's evaluation metadata lists the model's WildClawBench overall rank as 8 ^[7]. None of that tells you anything about serving cost, and SGLang's measurements are a separate set of numbers ^[8].

The compression story has two moving parts. NVFP4 stores key operations in a four-bit floating-point format aimed at NVIDIA's Blackwell generation, which cuts memory and compute at the cost of precision you then have to test for ^[9]. DFlash2 runs a smaller draft model that proposes tokens ahead of the 27B target, which verifies them; accepted proposals let the target emit several tokens per verification pass ^[10]. Gains depend on prompt, draft quality, hardware, batch size and memory configuration ^[11]. The recipe pulls a separate draft checkpoint, incoai/Qwen3.8-27B-DFlash2, and sets --speculative-num-draft-tokens 8 ^[12]. That is a second checkpoint and a second compatibility surface bolted onto an already hardware-specific stack ^[13].

Read the benchmark conditions before you read the headline number. SGLang says it validated the RTX 5090 and RTX PRO 6000 configurations with an 8,192-token input, a 1,024-token output and concurrency of one ^[14]. It reports 206.1 tokens per second on one RTX 5090 ^[1]. At concurrency one, that is roughly five seconds of decode for the 1,024-token output ^[22], a single-user latency figure rather than a throughput budget for a shared endpoint. SGLang has not published acceptance-rate or output-quality data for those runs ^[16], so the mechanism doing most of the work in the speculative path is unmeasured in public.

There is also a discrepancy worth resolving before anyone quotes the second number. The cookbook says the DGX Spark configurations booted and served under those settings while stating the project did not take DGX Spark throughput or acceptance-length measurements ^[15], and the same material reports 38.28 tokens per second on a DGX Spark ^[1]. Those two statements do not sit together ^[23].

The failure mode is already documented in the neighbourhood. A separate SGLang issue reported severely repetitive output from an unofficial NVFP4 conversion of Qwen3.8-27B because an FP8 scaling value for the output head was not loaded ^[17]. That concerns a different checkpoint from the RadixArk NVFP4 model in the cookbook and does not indicate a defect in the new recipe ^[18], but it is a clean illustration of why the exact checkpoint plus runtime pair is the unit of trust in mixed-precision serving.

Context on who is doing this work: SGLang is associated with Ying Sheng, whose January 2024 overview paired a structured language for model programs with RadixAttention for reusing cached prompt prefixes ^[19]. Sheng later co-founded RadixArk around SGLang with Banghua Zhu ^[20]. RadixArk reportedly launched with a $100M seed at a $400M post-money valuation, reported by TechCrunch, with Accel leading and Spark Capital plus angels including Intel CEO Lip-Bu Tan and xAI co-founder Igor Babuschkin participating ^[21].

What to watch: whether anyone outside the project reproduces 206.1 tokens per second on a 5090 with the cookbook's exact checkpoints, whether SGLang publishes acceptance rates and output-quality checks for the DFlash2 path, and whether the DGX Spark figure survives clarification. Until then, treat the number as a target to verify on your own hardware, not an input to a purchase order.