Cerebras answers the KV cache with DRAM stacked two generations out

Weights and cache load a memory system differently, and Cerebras' own framing says so: the memory occupancy of model weights amortizes across many inference sessions, while the KV cache for a long context belongs to one session alone ^[4]. Cache is therefore the term that scales with concurrency times context length, and it is the term a fixed on-wafer SRAM budget cannot absorb. GPU vendors bought their way out laterally, stacking HBM taller and putting more of it beside each accelerator ^[8]. That option is closed here. Wafer area is already fully spent on logic and memory, so any addition is a subtraction somewhere else, and Cerebras expects production to stay on 300mm wafers for the foreseeable future ^[5][6]. Up and out are the directions that remain.

Up is CS-6, the first wafer on which Cerebras will attempt DRAM stacked over logic and SRAM, and the company places it two generations out from the system it is announcing now ^[7]. That leaves one product generation in between ^[13]. Anyone sizing a deployment inside that window is sizing it against SRAM.

Out is Nexus, and it is worth reading what the rack actually relocates. The wafer's I/O leaves the compute module: two interchangeable modules attach to the wafer edges and carry RoCE v2 RDMA plus direct links to the other wafers in the rack ^[11]. Power delivery moves to the rack front, up to ten units per backpack, grouped so the operator picks a redundancy level ^[12]. None of that puts bytes closer to the compute. The rack-scale performance figure carried in the Tom's Hardware headline ^[18] transfers only if the limiter in your serving loop is something Nexus changes: wafer count per rack, the scale-up interconnect, or delivered power. If the limiter is cache capacity at your context length, the plumbing does not reach it.

The plumbing is good work, though. Mounting the wafer vertically lets Cerebras delete the PCB or substrate entirely and press a large copper busbar against the back of the die, cutting the losses a BGA part on a board pays on the way in ^[2]. And the claim for the stack is not only capacity: Cerebras says stacking preserves its inference lead while reducing the area the overall chip needs ^[15].

What the Hot Chips material does not contain is a byte. No per-wafer SRAM figure, and no capacity or bandwidth number for the planned DRAM layer ^[14]. Two generations is a long time to hold a niche on latency alone, and until those figures exist, the size of the gap being closed is a matter of trust rather than arithmetic.