The Neuron clocks NVIDIA's 74B Nemotron Bolt at 65.5 tokens per second on a laptop
The Neuron measured NVIDIA's Nemotron 3.5 Bolt 74B-A9B at 65.5 tokens per second on a laptop in a hands-on test. The build was heavily compressed, and the review's tool-assisted wins went to the hosted Super VL.
The Engineer · Build desk
Key figures: roughly 147 GB/s of implied weight traffic, a rough floor that ignores attention, KV cache and higher-precision layers; roughly 9 billion active parameters per token; 65.5 tokens per second measured on a laptop in The Neuron's review.
| Figure | Value | Note | Claim |
|---|---|---|---|
| Implied weight traffic at 2 bits per weight, roughly | 147 GB/s | rough floor; ignores KV cache | 17 |
| Active per token, roughly | 9 billion params | 4 | |
| Speed seen on laptop | 65.5 tokens/s | 1 |
What happened
- The local build was an IQ2_XXS GGUF run through Unsloth Studio on a Surface Laptop Ultra with an NVIDIA RTX Spark and 128GB of unified memory.
- The review credits Super VL, not Bolt, with the tool-assisted historical fact-check and the more successful first pass at a playable game.
- NVIDIA's Super VL blog claims up to 5x higher throughput than some open peers under its own agentic settings, which the review did not reproduce.
Why it matters
- What it limits constraint The 65.5 figure implies about 147 GB/s of memory traffic at 2 bits per weight, so it transfers only to laptops with comparable memory bandwidth.
- What doesn't add up contradiction NVIDIA designs Bolt to fit in 24GB, but the test ran on 128GB, so the speed result says nothing yet about the 24GB case.
- Who has to decide decision Teams weighing local models for agents still need their own tool-use test on the local build, because the review's tool-assisted result came from the hosted Super VL.
Bolt activates roughly 9 billion of its 74 billion parameters per token [4], or 12.2% [15]. If each active weight cost a flat 2 bits, a token touches about 2.25 GB of weights. At 65.5 tokens per second [1], that works out to roughly 147 GB/s of memory traffic [17]. This is a rough floor. It ignores attention, the KV cache and any layers kept at higher precision.
So the figure transfers to machines with memory bandwidth in that range. The review does not report the bandwidth of the Surface Laptop Ultra, which carried an NVIDIA RTX Spark and 128GB of unified memory [2]. A laptop with a narrower bus should decode more slowly.
The reviewer is open about the configuration. "This is an aggressively compressed local configuration. Its behavior should not be assumed to represent every precision, checkpoint, inference engine, or decoding configuration," the review says [3]. NVIDIA says Bolt is designed to fit in 24GB alongside other applications [6]. The test machine had more than five times that memory [16], so the run does not exercise the 24GB claim.
NVIDIA pitches Bolt for local agents that search, use tools and write or debug code [13]. Both models answered basic logic and business-analysis questions well [10]. The tool-assisted historical fact-check and the more successful first pass at a playable game are credited to Super VL [11]. Super VL ran in OpenRouter Chat, and the reviewer did not select or verify the GPU, the provider route or the served weight precision [9]. Bolt's runs were text-only [12].
The reviewer wrote: "I can run a remarkably capable model on my own machine, at interactive speeds, without sending the inference workload to a remote provider." [14] The speed half of that sentence is measured. In our view the 65.5 figure supports one conclusion: a 2-bit-class 74B mixture-of-experts model decodes at interactive speed on 128GB-class hardware. Saying more about agents needs a tool-use loop run on the local build.
NVIDIA's own throughput claim has the same transfer problem. It says Super VL reaches up to 5x higher throughput than some open peers under its measured agentic settings, and the reviewer notes these are NVIDIA's claims, not results from the review's tests [18]. The number holds for a workload that matches those settings.
What to watch
- Bolt E4B, the 4GB-footprint model NVIDIA slated for October 23, which would give a smaller local build to test the same way.
- A tool-loop run of Bolt on local hardware, ideally on a 24GB machine, to check both agent reliability and NVIDIA's fit claim.
- A dedicated vision benchmark for Bolt and Super VL, which the review says neither has had.
Clarity's read
What the record supports and how the coverage leans. The claims behind it follow.
Reality
- Evidence45
- Adoption
- Not scored
- Hype gap+15
- Incentives
- Not scored
- Confidence50
Claim ledger
Ranked by verification strength, evidence, and original report placement.
- [1]
The reviewer watched NVIDIA Nemotron 3.5 Bolt 74B-A9B generate at 65.5 tokens per second on a laptop.
- [2]
The local setup was Bolt 74B-A9B as an IQ2_XXS GGUF, running through Unsloth Studio on Microsoft's Surface Laptop Ultra with NVIDIA RTX Spark and 128GB unified memory.
- [3]
The review calls this an aggressively compressed local configuration and says its behavior should not be assumed to represent every precision, checkpoint, inference engine, or decoding configuration.
- [4]
Bolt 74B-A9B has 74 billion total parameters with roughly 9 billion active per token, in a mixture-of-experts architecture.
- [5]
The 24GB Bolt release relies on native 2-bit quantization and quantization-aware techniques.
- [6]
NVIDIA says Bolt is designed to fit in 24GB alongside other applications.
- [7]
NVIDIA announced the models on October 15, positioning Super VL for specialized always-on multimodal agents and Bolt for agentic work on local NVIDIA hardware.
- [8]
A smaller Bolt E4B model, designed around a 4GB footprint, is slated for October 23.
- [9]
Super VL was tested in OpenRouter Chat; inference ran on a remote provider's infrastructure, and the reviewer did not select or verify the underlying GPU, exact provider route, or served weight precision.
- [10]
Both models answered the reviewer's basic logic and business-analysis questions well.
- [11]
Super VL was particularly effective in a tool-assisted historical fact-check and made a more successful first pass at producing a playable game.
- [12]
The reviewer's Bolt experiments were text-only, and neither model has undergone a dedicated vision benchmark in the review.
- [13]
NVIDIA's advertised target for Bolt is everyday local agents that can search, use tools, and write or debug code.
- [14]
"I can run a remarkably capable model on my own machine, at interactive speeds, without sending the inference workload to a remote provider."
- [15]
About 9 billion active parameters out of 74 billion total is roughly 12.2% of Bolt's parameters per token.
- [16]
The test machine's 128GB of unified memory is more than five times the 24GB NVIDIA designs Bolt to fit in.
- [17]
At a flat 2 bits per weight, 9 billion active parameters is about 2.25 GB of weights per token; at 65.5 tokens per second that implies roughly 147 GB/s of weight traffic, a rough floor that ignores attention, KV cache and higher-precision layers.
- [18]
NVIDIA's Super VL technical blog says the model can achieve up to 5x higher throughput than some open peers under NVIDIA's measured agentic settings; the reviewer says these are NVIDIA's benchmark and deployment claims, not the performance measured in the review's tests.
Get this ledger inside ChatGPT. Ask about any of our stories and the answer brings its figure, each claim’s verdict and the sources. Free, no account.
Sources
1 independent publisher whose own reporting we read for this story.
- theneuron.aiI Tested NVIDIA Nemotron Bolt and Super VL. Here’s Where Each Shines.
1 article · October 16, 2026
Topics and entities
Follow any of these and your For You feed starts watching them — no settings page required.
Topics
Entities
- MicrosoftFollow
- The NeuronFollow
- Nemotron 3.5 Super VLFollow
- NvidiaFollow
- OpenRouterFollow
- Unsloth StudioFollow
- Nemotron 3.5 BoltFollow