NVFP4 squeezes Qwen3.8's 2.4 trillion weights onto eight B300s at 150 GB a GPU

Two point four trillion parameters at four bits apiece is 1.2 TB, which is exactly the footprint the community NVFP4 build reports ^[2][6][15]. That match means the checkpoint sits at the arithmetic floor for W4A4, so there is no slack for a layer someone decided to leave in higher precision. Divide across the node and the weights alone claim 150 GB per GPU ^[14]. KV cache, activations and multi-token-prediction draft state all live in what is left.

That is where the attention split stops being an architecture footnote. One layer in four runs full quadratic attention, 23 of 92, while the other 69 are Gated DeltaNet layers with a bounded recurrent state instead of a growing cache ^[7][16]. A 262K native window extensible to 1M is a memory claim before it is a quality claim ^[3], and only a quarter of the stack is paying the per-token memory tax.

The MoE side is cheaper than the headline number suggests and more expensive than the activated one. About 4 percent of parameters fire per forward pass ^[17], and serving compute tracks that count ^[8], but capacity does not. All 512 experts stay resident, which is the 1.2 TB you just divided by eight ^[8].

AWS frames the trade-off as operational, with no per-token API fees at scale on one side and purpose-built GPU infrastructure on the other ^[11]. The text does not close the loop with a number: there is no hourly rate for the ml.p6-b300 instance and no break-even against API pricing ^[18]. The unit of cost here is an eight-GPU Blackwell Ultra node held for as long as the endpoint exists ^[4], not a token. Utilization sets the break-even, and a node at a 10 percent duty cycle bills the same as a saturated one.

The benchmark scores are Alibaba's own, and the post does not tie them to the quantized artifact it actually deploys ^[10][19]. For PaperBench 93.0 or 86.6 on terminal coding to transfer to your endpoint, several things have to hold: W4A4 has to be behavior-preserving on your prompt distribution, your `reasoning_effort` setting has to match whatever produced the table ^[9], and MTP acceptance has to be high enough that speculative decoding earns back the draft compute ^[5]. Those are a validation run, not something readable off a table.

The least glamorous part of the guide is the part that is doing real work. HyperPod puts EKS in front of driver installation, model download, health monitoring, autoscaling and node replacement ^[12], because a single node holding 1.2 TB of weights is also a single failure domain, and recovery means pulling that 1.2 TB again. Kimi K3 got the same treatment in the first post of the series ^[13].</body_markdown> </invoke>