NVIDIA's Bolt packs 74B parameters into a claimed 24GB by storing them at 2 bits
NVIDIA says its Nemotron 3.5 Bolt 74B-A9B agent model fits in 24GB and will ship with open weights, data and recipes. The footprint, the license terms and the quality claims stay unverified until the weights are live and outside testers have run them.
The Engineer · Build desk
Key figures: 74B parameters stored at 16 bits take about 148GB. Stored at a flat 2 bits they take about 18.5GB. NVIDIA's claimed 24GB is about 5.5GB above that flat 2-bit layout.
| Figure | Value | Note | Claim |
|---|---|---|---|
| 74B parameters stored at 16 bits, about | 148 GB | 13 | |
| At a flat 2 bits, about | 18.5 GB | 14 | |
| Gap to claimed 24GB, about | 5.5 GB | 16 |
What happened
- The release was scheduled for October 15, and the dev.to post covering it says the weights were not yet confirmed live when it was written.
- NVIDIA named llama.cpp, vLLM, Ollama, Unsloth and LM Studio as launch runtimes, plus the Hermes and OpenClaw agent frameworks.
- NVIDIA calls Bolt faster than dense models and smarter than same-footprint MoE models, and the post says no independent benchmarks exist.
- The closed models launched this month are priced at $4/$20 (Muse Wonder) and $2/$10 (GPT-6 Sol, Gemini 4 Argon intro) per million tokens.
Why it matters
- cost Bolt trades per-token fees for hardware and electricity, but NVIDIA has announced no retail price for RTX Spark, so a break-even against the closed APIs cannot be calculated yet.
- decision A team cannot commit a product to Bolt until the license text is published, and it should pull only from the official listing name because spoofed listings appear fast.
- capability If the release matches the announcement, teams can fine-tune on their own data, run fully offline, and inspect the training data and recipes.
- constraint NVIDIA's PAIR router sends whole jobs to one machine and does not pool memory, so every node that takes a Bolt job needs room for the whole model.
Mixture-of-experts saves compute, not storage [3]. Bolt works on about 9B of its 74B parameters for any one token [3]. That is roughly 12% of the total [12]. Compute per token looks like a 9B model. Memory looks like a 74B model, because the inactive parameters still have to be stored [3].
NVIDIA's route to a small footprint is 2-bit quantization with quantization-aware training. The model is trained to expect the compression instead of being squeezed after the fact [4]. That is the right way to attempt it, in our view.
The sizes do not quite close. At the usual 16 bits, 74B parameters take 148GB [13]. At a flat 2 bits they take 18.5GB [14]. NVIDIA claims 24GB [1], which works out to about 2.6 bits per parameter [15]. That leaves 5.5GB beyond a pure 2-bit layout [16]. The dev.to post covering the announcement does not say what fills it, so the model card's per-tensor breakdown is the document to read.
The 24GB figure is also a statement about the weights alone. According to the post, context length, the KV cache, runtime overhead and agent software all add memory, and it advises treating 24GB as the floor [5]. A machine with exactly 24GB free has nothing left for any of them.
On quality, the transfer condition is plain. For the "smarter than same-footprint MoE models" half of NVIDIA's line to hold on your workload, the 2-bit weights would have to hold up on your own tasks. The only test of that so far is NVIDIA's [17]. Bolt has been benchmarked by its maker and not yet confirmed downloadable [2].
In our view, publishing the training data and recipes next to the weights is what makes the 24GB claim checkable [1]. It is checkable only if the release matches the announcement.
What to watch
- Whether the weights appear under an official Hugging Face or Ollama listing, and what license text ships with them.
- Independent benchmarks of the 2-bit model against the closed models and against same-footprint MoE models.
- The model card's per-tensor breakdown for the gap between 18.5GB and 24GB, and measured memory at real context lengths.
Clarity's read
What the record supports and how the coverage leans. The claims behind it follow.
Reality
- Evidence25
- Adoption
- Insufficient
- Hype gap+30
- Incentives
- Insufficient
- Confidence40
Claim ledger
Ranked by verification strength, evidence, and original report placement.
- [1]
NVIDIA announced Nemotron 3.5 Bolt 74B-A9B on October 13 as a 74-billion-parameter agent model it says fits in 24GB of memory, with the weights, the training data and the training recipes all to be published.
- [2]
The release was scheduled for October 15, one day before the RTX Spark laptops go on sale; as of the post's writing the weights were not yet confirmed live, and all coverage so far was pre-launch.
- [3]
Bolt is a sparse mixture-of-experts model: only about 9B of its 74B parameters work on any one token, so compute per token looks like a 9B model; inactive parameters still have to be stored, so mixture-of-experts saves compute, not storage.
- [4]
The storage approach is 2-bit quantization with quantization-aware training: parameters are stored at 2 bits instead of the usual 16, and the model is trained to expect the compression rather than squeezed after the fact.
- [5]
24GB is NVIDIA's claim about the weights, not a full system requirement; context length, the KV cache, runtime overhead and agent software all add memory, and the post says to treat 24GB as the floor.
- [6]
NVIDIA named five runtimes at launch, llama.cpp, vLLM, Ollama, Unsloth and LM Studio, plus the Hermes and OpenClaw agent frameworks.
- [7]
The exact license terms are still to be confirmed because the weights were not public at announcement, and wrong or spoofed model listings appear fast around popular releases, so the official listing name should be checked.
- [8]
The closed models launched this month are priced per million tokens: Muse Wonder at $4/$20, GPT-6 Sol at $2/$10, and Gemini 4 Argon at $2/$10 introductory pricing.
- [9]
RTX Spark laptops, with up to 128GB unified memory, go on sale October 16, and NVIDIA has announced no retail prices; Bolt itself is free to download, so the user pays for hardware and electricity.
- [10]
PAIR routes whole inference jobs to one machine and does not pool GPU memory, so three PCs with 24GB each do not become one 72GB virtual GPU.
- [11]
Open weights let users fine-tune the model for their own agents, run it entirely offline, and inspect the training data and recipes.
- [12]
Active parameters are roughly 12% of the total.
- [13]
74B parameters stored at 16 bits take about 148GB.
- [14]
74B parameters stored at a flat 2 bits take about 18.5GB.
- [15]
NVIDIA's 24GB claim for 74B parameters works out to about 2.6 bits per parameter.
- [16]
The claimed 24GB is about 5.5GB above a flat 2-bit layout of 74B parameters.
- [17]
NVIDIA's line that Bolt is 'faster than dense models, smarter than same-footprint MoE models' is NVIDIA's own claim; the post says there are no independent benchmarks for Bolt 74B, and that closed models still lead on independent benchmarks.
Get this ledger inside ChatGPT. Ask about any of our stories and the answer brings its figure, each claim’s verdict and the sources. Free, no account.
Sources
1 independent publisher whose own reporting we read for this story.
- dev.toNVIDIA Just Open-Sourced a 74B Agent Model That Fits on a Laptop
1 article · October 14, 2026
Topics and entities
Follow any of these and your For You feed starts watching them — no settings page required.