Science1 publisher2 min readPublished
NVIDIA's Nemotron 3 Super routes each token through a tenth of its 120 billion parameters
NVIDIA says multi-agent systems burn up to 15 times the tokens of a standard chat, and Nemotron 3 Super is its open-weight attempt to make each of those tokens cheaper to produce. The efficiency figures come with NVIDIA's own hardware and its own predecessor as the baselines.
The Scientist · Science desk
What happened
- NVIDIA released Nemotron 3 Super, a model with 120 billion total parameters and 12 billion active, aimed at multi-agent work such as software development and cybersecurity triaging.
- The model ships with a native one-million-token context window, which NVIDIA says gives agents long-term memory across a long task.
- Weights, datasets and training recipes are published, so developers can customize the model and deploy it on their own infrastructure.
- On PinchBench, a new benchmark that measures how well a model performs as the brain of an OpenClaw agent, Super scores 85.6 percent across the full suite, which NVIDIA calls best in its open-model class.
- Post-training used reinforcement learning across 21 environment configurations in NeMo Gym and NeMo RL, with more than 1.2 million environment rollouts.
Compiled by The ScientistSomething wrong?How this is made
Why it matters
- cost NVIDIA's own estimate is that multi-agent systems consume up to 15 times the tokens of a chat session, so per-token compute is the line item that grows fastest in an agent deployment, and a model that touches a tenth of its weights per token moves that line more than a leaderboard position does.
- constraint Capturing the claimed 4x inference speedup means buying Blackwell, because the figure is NVFP4 on a B200 against FP8 on an H100; a team on H100s is choosing this model on other grounds.
- decision The 5x throughput claim is baselined on NVIDIA's own previous Super, so anyone weighing this model against a different vendor's has to run that comparison in-house.
- precedent NVIDIA shipped the model alongside a benchmark it introduces in the same post. Until someone else reproduces PinchBench, the vendor sets the terms of comparison for agent-brain models.
The routing layer is the part NVIDIA changed. In a standard mixture-of-experts model, tokens travel from the model's full hidden dimension to the experts, and NVIDIA says that path becomes a bottleneck as models grow, raising compute cost and capping how many experts can practically be deployed [10]. Super compresses tokens before they reach the experts, which NVIDIA says lets it call four times as many specialists at the same inference cost [9]. Only a subset of experts fires on any given token [18]. At the published sizes, that subset is a tenth of the model [19].
The rest of the stack splits work by layer type. Mamba-2 layers do most of the sequence processing, and NVIDIA credits their linear-time complexity in sequence length for making a million-token window work in practice [16]. Attention layers are interleaved at key depths, because pure state space models can struggle with precise associative recall, the find-one-fact-in-a-long-document case [17]. Multi-token prediction predicts several future tokens in a single forward pass and gives the model built-in speculative decoding [11].
One of the efficiency numbers moves two variables at once. The claimed 4x inference speedup is NVFP4 on a B200 measured against FP8 on an H100, which NVIDIA says holds accuracy [13]. That changes the number format and the GPU generation together.
The PinchBench result scores task performance as the brain of an OpenClaw agent [15]. That is not the same measurement as tokens consumed per finished task, and the announcement does not report per-task token counts or costs. If the problem is what NVIDIA calls the thinking tax, the cost of running a large reasoning model on every sub-task [4], then cost per completed task is the figure that settles it.
The 15x token multiplier that frames the whole release is NVIDIA's own figure for multi-agent systems against standard chats, driven by re-sending history, tool outputs and reasoning steps every turn [3]. NVIDIA also describes a related failure it calls context explosion, where agents on long tasks drift away from the original objective [5]. Weights, datasets and recipes are published for deployment on a developer's own infrastructure [8]. A team already running agents can measure its own multiple, and its own cost per finished task.
What to watch
- An independent PinchBench run by someone other than NVIDIA, on the same full test suite, would show whether 85.6% survives a different harness.
- A published tokens-per-completed-task comparison against the previous Nemotron Super would put a checkable cost figure behind the 5x throughput claim.
- Long-context accuracy results at or near the full million tokens, from outside NVIDIA, would test whether the interleaved attention layers hold retrieval at length.