Topic

State Space Models

Sequence models that carry a fixed-size recurrent state through a sequence instead of attending over every previous token, so memory use does not grow with context length.

Current clusters

build1 publisher

A 30B model that activates 3B still has to keep all 30B in VRAM

NVIDIA's dense-versus-MoE explainer uses Nemotron 3.5 Lightning to walk through per-layer routing. Its own text puts the throughput number and the memory number on two different parameter counts.

Publishers:developer.nvidia.com

Reality

Evidence58
Adoption20
Hype gap+15
Incentives82
Confidence62