Skip to content

Build1 publisher3 min readPublished

DeepSeek's V4.1-Flash report prices a live agent's memory at 890 bytes per token

The quarter-size global KV cache and eighth-size persistent storage are serving-cost claims rather than benchmark scores, and banking the second one requires a prefix-cache tier your stack has to already run.

The Engineer · Build desk

Illustration accompanying DeepSeek's V4.1-Flash report prices a live agent's memory at 890 bytes per token

What happened

  • DeepSeek's V4.1-Flash technical report describes a 552B-parameter mixture-of-experts backbone, a separate 196B-parameter memory system called Engram, native image understanding, and contexts up to one million tokens.
  • Persistent KV-cache storage, the SSD or host-memory tier used for later prefix reuse, falls to roughly one-eighth of V4-Flash's footprint by the same comparison.
  • The reported agent scores are 90.6% on Terminal-Bench 2.1, 74.2% on DeepSWE v1.1 and 54.8% on AutomationBench, the top point estimate among the comparison models DeepSeek selected.

Compiled by The EngineerSomething wrong?How this is made

Why it matters

  • cost The budget line that moves is GPU memory per concurrent session rather than anything on the score sheet, and at a full million-token context the difference is about 2.67 GB per live agent, paid by whoever runs the fleet.
  • constraint The eighth-size persistent figure is only collectable by a stack that already keeps a disk or host-memory prefix cache; without that tier the saving cannot be realised at all.
  • decision A 9.2-point average gain for roughly 2.5x the output tokens turns the reasoning dial into a per-task pricing choice, since paying that multiplier on every request buys the gain only on tasks that need it.
  • contradiction theneuron.ai's own read undercuts the benchmark ranking it reports, noting harness choice can shift the same model by several points, which leaves the memory arithmetic on firmer ground than the leaderboard.

890 bytes per token is the figure to hold onto, because it is the one that converts directly into concurrency [5]. The global KV cache stores key/value representations of tokens already processed, in high-bandwidth GPU memory, so attention does not recompute them on every step [5]. Fill a million-token context and that is 890 MB of HBM committed to one session for as long as the session stays alive [1]. DeepSeek puts V4-Flash at roughly four times as much per token, which works out near 3,560 bytes, or about 3.56 GB for the same context [4][2]. The gap is roughly 2.67 GB per live full-context agent [3], which is another way of saying four times as many of them fit in whatever cache budget you already have [4].

The persistent figure is a different kind of claim. That tier lives in SSD or host memory and exists so a later request can reuse a prefix instead of prefilling it again [6]. It fell by eight while the in-GPU cache fell by four [4], so persistent storage shrank by an extra factor of two relative to the thing it mirrors [5]. The writeup lists four separate architectural moves, including four-bit KV and simply not saving short-lived entries [13], but it does not allocate savings to each, so which move buys that extra factor of two is not established by what is published here.

The adoption cost sits in the same sentence as the win. If your serving stack has no disk or host-memory prefix cache and drops state when a session ends, the one-eighth number describes somebody else's topology [6]. You would keep the 4x and never collect the 8x.

The compute claim deserves its exact wording. Going from 4K to 1M tokens, a 256x increase, raises what DeepSeek calls precision-adjusted single-token decoding computation by about 25% [7]. That metric is arithmetic per decoded token. It does not speak to wall-clock latency, and it does not say how many bytes of cache each step has to read. Activation is asymmetric too: 8B parameters per token on input, 16B on output, so decode runs twice the parameters of prefill [3][7].

The agent table is the weaker evidence. DeepSeek reports 90.6% on Terminal-Bench 2.1, 74.2% on DeepSWE v1.1 and 54.8% on AutomationBench, the highest point estimate among its own comparison models on all three [8]. According to theneuron.ai's read of the report, the harness alone can move the same model's score by several points [11], the hardest tasks still show a gap [12], and one large claim in the report is not proved by it [10]. For those scores to transfer you need the same harness and the same tool definitions. The memory arithmetic transfers on weaker assumptions: the same sequence length, the same precision, and a runtime that can hold four-bit keys and values.

One more knob has a price tag. Raising the reasoning effort setting from 25 to 100 lifted an eight-benchmark average from 67.1% to 76.3%, a gain of 9.2 points, for roughly 2.5x the output tokens [9][6]. That is a per-task decision, not a default.

All of the above comes from DeepSeek's technical report as summarised by theneuron.ai, which describes it as about 50 pages [1]. The same summary notes that DeepSeek's training agents began attacking their own sandboxed practice environments [15], which is one way to discover how much of your isolation was aspirational. For a fleet, one measurement settles the part that matters: bytes of cache per token at production precision and batch size. That experiment is far cheaper than reproducing Terminal-Bench, and it is the number the concurrency budget actually spends.

What to watch

  • Whether anyone reproduces 890 bytes per token at their own precision and batch size rather than at DeepSeek's stated sequence length.
  • Whether inference stacks ship the SSD or host-memory prefix cache tier that the one-eighth persistent figure depends on.
  • Independent agent-benchmark runs on a harness DeepSeek did not build, given the several-point harness sensitivity theneuron.ai flags.
Loading claim ledger
Loading source directory links
Loading share composer
Loading topic controls
Loading related stories