Kimi K3's real gate is 1.4TB of VRAM and a bespoke licence, not engineering

Moonshot AI published the weights for Kimi K3, a 2.8-trillion-parameter model, on 27 July 2026, alongside production inference support ^[1]. The engineering of serving it is achievable; according to the dev.to teardown of the deployment, it is the arithmetic that defeats projects, and it does so quietly, several months after the budget is approved ^[2].

Start with the weights, because every other requirement follows from them. At MXFP4, roughly four bits per parameter, 2.8 trillion parameters occupy about 1.4 TB, and all of it has to be resident in accelerator memory to serve requests at reasonable speed ^[3]. Then the key-value cache: the model advertises a million-token context, and attention state for every concurrent request scales with both context length and batch size ^[4]. Published vLLM metadata puts the minimum viable serving footprint at roughly 1,680 GB ^[5], which leaves on the order of 280 GB above the weights for cache and overhead ^[1].

That figure eliminates the configuration most teams have already sketched. Eight H100s at 80 GB provide 640 GB, less than half what the weights alone require ^[6]. Put differently, a full eight-way H100 node covers about 38 percent of the minimum serving footprint ^[2], and reaching 1,680 GB with 80 GB cards takes three such nodes on paper ^[3], which converts a memory shortfall into an interconnect problem.

Interconnect is where the architecture bites. Kimi K3 is a mixture of experts that routes each token to sixteen of 896 experts, about 1.8 percent of them ^[7][4], which generates substantial all-to-all traffic between the devices holding different experts ^[7]. A box with adequate total memory and weak interconnect will deliver throughput far below the specification sheet, and diagnosing that after purchase is an expensive lesson ^[8].

The shapes that actually work: eight current-generation 288 GB accelerators, NVIDIA B300 or AMD MI355X, give around 2.3 TB in a single node ^[9], roughly 620 GB above the minimum footprint for batching ^[5]; sixteen B200 or GB200-class cards reach a similar total across a larger footprint ^[10]; and for sustained production rather than a proof of concept, Moonshot's own guidance points at supernodes of sixty-four or more accelerators ^[11].

Before any of that, the licence. The weights carry a custom grant the model card names the Kimi K3 License, not a plain MIT or Apache grant, and third-party summaries describing it as one should not be relied upon ^[12]. The specific things to read for are attribution requirements and conditions that attach at particular scales of use ^[13]. That review is an afternoon of work and it determines whether the hardware conversation is worth having.

What the weights buy is control: data never leaves the estate, nobody deprecates the model or changes the price, and you can fine-tune, quantise further or modify serving behaviour in ways an API will not permit ^[14]. What they do not buy is a cheaper way to do what the API already does ^[15].

The software side is more settled. The model card lists vLLM, SGLang and TokenSpeed as supported engines ^[16], and support for Kimi Delta Attention shipped alongside the weights rather than arriving later, so a current vLLM build has the kernels while an older installation does not, which is the first thing to check when a deployment refuses to start ^[17]. Then plan the logistics of pulling and storing over a terabyte of weights ^[18].

Three things to watch on any K3 plan: which licence conditions trigger at your usage scale ^[13], whether the interconnect story matches sixteen-of-896 routing rather than just total VRAM ^[7], and the exact engine version in the image ^[17].