Kimi Linear's 75% KV cache cut comes from making three of every four layers recurrent

The 75% cache figure follows from the layer count. According to a dev.to write-up of the paper, Kimi Linear stacks three Kimi Delta Attention (KDA) layers for every Multi-Head Latent Attention (MLA) layer ^[6]. A KDA layer keeps a fixed-size memory matrix and updates it token by token, so its state does not grow with the sequence ^[5]. Only the MLA layer stores keys and values per token. With one layer in four holding a growing cache, the stack keeps a quarter of an all-MLA cache. The reported cut is 75% ^[1]. The KDA state is a fixed cost, so the saving gets close to 75% at long context and is smaller on short prompts ^[5][1].

The baseline matters. MLA, the attention used in DeepSeek models, already compresses keys and values into a low-dimensional latent space ^[7]. Both the 75% and the 6.3x decoding figure are measured against full MLA ^[15]. The cut therefore comes on top of a cache that was already compressed.

The recurrent layers are where the engineering is. Earlier linear variants such as GLA and Gated DeltaNet gated whole heads, while KDA gives each feature channel its own decay scalar ^[3]. Each step decays the state per channel, removes the old state's projection onto the current key, and writes the new key-value pair ^[4]. A fixed-size memory overwrites older information ^[5]. The MLA layer at every fourth position supplies the global coherence that pure recurrent models struggle to keep ^[16]. Position comes from the same decay gates, and the model uses no positional encoding ^[9].

Speed depends on the kernels. The authors split sequences into 256-token chunks and use a WY representation, so batches of rank-1 updates become triangular solves on Tensor Cores ^[8]. It is careful work, and it is what the throughput numbers measure. Because the checkpoints and kernels are open ^[1], anyone can rerun those numbers.

The speed rows scale with context. At 4k tokens on MMLU-Pro the model matches MLA at equivalent speed ^[11]. At 128k it is 3.98x faster on RULER ^[12]. Prefill is 2.3x faster at 512k ^[14], and decoding is 6.3x faster at 1M ^[15]. The write-up's headline says the design beats full attention, though on the 4k row the verb the numbers support is "matches." The 94.8 on RULER at 1M ^[13] is the row that tests whether one exact-attention layer in four keeps long-range recall.

Several conditions have to hold before those multiples show up on someone else's serving bill. The traffic has to run at hundreds of thousands of tokens. The serving stack has to run the chunked kernels. The model also has to be built this way. The released checkpoint is a sparse MoE with 48B total parameters and 3B activated per token ^[10]. The saving comes from the layer layout ^[6], so a full-attention checkpoint already in production does not get it.

In my view the cache claim holds on structure alone. The throughput figures are Moonshot's own measurements, and the dev.to write-up does not say what hardware or batch size produced them.