A 284B model at 25 tokens a second on one 5090, and 192 GiB of DDR5 doing the quiet part

The sparsity is what makes the claim plausible before anyone checks it. DeepSeek-V4-Flash routes each token through six of 256 experts in each of 43 layers, with roughly 13 billion of 284 billion parameters active ^[7]. That is about 4.6 percent of the weights per token ^[1] and about 2.3 percent of the experts in any given layer ^[4]. The paper says the active computation fits inside the 5090's 32 GB of VRAM while the full expert pool stays much larger ^[8], so the hard part is not arithmetic. It is fetching: which experts sit in VRAM, and what to do when the router asks for one that does not.

FreeToken's answer is a global least-recently-used expert cache, with cache misses split between PCIe transfer and CPU execution according to the bandwidth available at that moment ^[10][9]. Prompt processing uses full-layer double buffering, staging the next layer while the GPU works the current one ^[10].

Which is why Table 1 is the more informative page. Alongside the 32 GB card sit 192 GiB of DDR5 and a Ryzen 9 9950X3D ^[6], roughly six times as much memory outside the GPU as inside it ^[2]. The GLM-5.2 figure makes the same point from the other direction: 14.9 tokens per second on a single 96 GB RTX PRO 6000 ^[5], a card with three times the 5090's VRAM, on a different model. Card capacity is not the variable this work is reporting.

Then the metric. The paper labels it decode throughput, per-request mean tokens per second, and does not state elapsed time from prompt submission through completion ^[11]. At 22 to 25 tokens per second, a 1,000-token reply is 40 to 45 seconds of decode alone ^[3], before prefill. The project's own agent machinery suggests prefill is not cheap: semantic-aware checkpointing at thinking segments, tool calls and conversation turns exists so the engine can re-prefill only a new suffix after an edit ^[14], and the GPU expert cache can be rebuilt without reloading the host-resident pool ^[15]. Features built to avoid re-prefilling concede that prefill costs something the headline number omits.

The comparison set, six GPU systems over four workloads against llama.cpp, Ollama and KTransformers in sections 5.1 and 5.2, was run by the authors ^[16], and the benchmarks have not been independently replicated ^[13]. The roster, Berkeley's Sky Computing Lab plus Keutzer, Han, Zaharia and Stoica ^[17], is why the preprint will be read this week. It is not the same thing as a second machine producing the same tokens per second.