Unsloth's figures suggest a 128 GB Mac can run NVIDIA's Nemotron 3.5 Super VL with vision
Nemotron 3.5 Super VL, NVIDIA's 120B model, had GGUF, MLX, FP8 and EXL3 conversions on Hugging Face a day after its October 15 release. The roughly 128 GB memory figure comes from a DEV Community write-up whose author did not run the model.
The Engineer · Build desk
Three memory figures. The rapid-mlx card measured a peak of 72.7 GB at 1K context on an M3 Ultra. The NVFP4 repo is 75.1 GB. Going by Unsloth's numbers, llama.cpp should run the model with vision on a 96 GB workstation card or a 128 GB Mac.
| Figure | Value | Note | Claim |
|---|---|---|---|
| Peak measured by rapid-mlx, M3 Ultra, 1K context | 72.7 GB | 9 | |
| NVFP4 repo size | 75.1 GB | 6 | |
| Workstation card for vision in llama.cpp, per Unsloth | 96 GB | or a 128 GB Mac | 20 |
What happened
- The headline checkpoint is NVFP4, a 4-bit format only Blackwell executes natively, and the model card lists DGX Spark, one H200, two H100s or one GB200 as targets.
- Going by Unsloth's numbers, llama.cpp should run the model with vision on a 128 GB Mac or a 96 GB workstation card, according to the DEV Community write-up.
- The only measured memory figure in the write-up is the rapid-mlx card's 72.7 GB peak, taken on an M3 Ultra at a 1K context.
Why it matters
- What it limits constraint A single consumer card cannot hold the 120B at any bit width, so RTX 5090 owners are pointed to NVIDIA's 2-bit Bolt 74B-A9B for 32 GB+ GPUs.
- Who's exposed exposure Browser-agent builders have a number to check: NVIDIA's own NVFP4 run is 11 points down on WebVoyager, per the write-up, so any quant choice should be tested on that task.
- Who has to decide decision A Mac owner who wants vision today picks llama.cpp with the mmproj file, and so works from Unsloth's planning figure instead of the one measured peak.
In our view, memory decides whether the model loads and Blackwell decides how fast it runs. The card says inference "uses W4A4 on Blackwell and W4A16 on non-Blackwell platforms" [4]. On a Hopper card the checkpoint still loads, but the matmuls run with 16-bit activations [5]. The write-up's summary is that you get the memory savings without the FP4 tensor-core speed [5].
The format is also not 4 bits throughout. BF16 is kept for the vision encoder, the connector, the Mamba convolutions and the embeddings, and FP32 for the MoE router [7]. A flat 4 bits across 120B parameters would come to 60 GB [19]. The actual checkpoint is 75.1 GB across 88 safetensors shards [6].
The 128 GB figure is a planning number. The author says the memory estimates are each source's own and that none of them were measured by the author [8]. The one measured peak, 72.7 GB, comes from an MLX build with no image input and no MTP [10], at a 1K context [9]. The card supports up to 1M tokens [11]. Against 128 GB, that peak leaves 55.3 GB [21], and we'd expect long context and images to use some of it.
The scores on the card are NVFP4 scores, and SWE-bench Verified is 71.07 [16]. Nemotron 3 Super was scored with NeMo Evaluator and NeMo Skills, and 3.5 with NeMo Gym, so the author treats the deltas as directional [15]. For that number to transfer to a GGUF or EXL3 build, the build would have to lose no more than NVFP4 does on NVIDIA's harness. NVIDIA's own BF16 and NVFP4 runs used identical harnesses, and most rows land within noise, with a few exceptions [14].
The card's DGX Spark Quick Start runs vllm/vllm-openai:v0.32.0 with `--max-num-seqs 1` and `--gpu-memory-utilization 0.75` [17]. That is one concurrent sequence and a 75% memory cap for vLLM, a single-user setup.
What to watch
- A measured memory peak for the llama.cpp build with vision on a 128 GB Mac or a 96 GB card would confirm or move the 128 GB figure.
- The rapid-mlx card moving 96 GB Macs from unverified to verified, or enabling image input and MTP.
- Community WebVoyager and SWE-bench runs on the GGUF, FP8 and EXL3 builds, set against NVIDIA's BF16 and NVFP4 rows.
Clarity's read
What the record supports and how the coverage leans. The claims behind it follow.
Reality
- Evidence55
- Adoption
- Not scored
- Hype gap+15
- Incentives
- Not scored
- Confidence55
Claim ledger
Ranked by verification strength, evidence, and original report placement.
- [1]
NVIDIA released Nemotron 3.5 Super VL on October 15, a 120B vision-language MoE with 12B active parameters.
- [2]
A day after release there were GGUF, MLX, FP8 and EXL3 conversions of the model on Hugging Face.
- [3]
The headline checkpoint is NVFP4, a 4-bit format only Blackwell executes natively, and the model card lists four deployment targets: DGX Spark, one H200, two H100s, or one GB200.
- [4]
The model card says inference "uses W4A4 on Blackwell and W4A16 on non-Blackwell platforms".
- [5]
A Hopper card still loads the small checkpoint but runs its matmuls with 16-bit activations, so you get the memory savings without the FP4 tensor-core speed.
- [6]
The NVFP4 repo is 75.1 GB across 88 safetensors shards.
- [7]
In the NVFP4 checkpoint the vision encoder, connector, Mamba convolutions and embeddings stay in BF16, and the MoE router stays in FP32.
- [8]
The author has not run the model. The memory estimates are each source's own planning figures, and the author has not measured any of them.
- [9]
The rapid-mlx card reports a measured peak of 72.7 GB at 1K context on an M3 Ultra.
- [10]
The rapid-mlx card says 96 GB Macs are not yet verified and that image input and MTP are not enabled.
- [11]
The model supports up to 1M tokens of context, with native Multi-Token Prediction for speculative decoding.
- [12]
A single consumer card cannot hold the 120B model at any bit width, and the RTX 5090 is the one popular card left out of the run-it list.
- [13]
NVIDIA's answer for smaller cards is Nemotron 3.5 Bolt 74B-A9B 24GB, compressed from Super VL with native 2-bit quantization and aimed at llama.cpp on 32 GB+ discrete GPUs (Ada Lovelace and Blackwell).
- [14]
NVIDIA ran BF16 and NVFP4 with identical harnesses and most rows are within noise, but a few are not; the author flags an 11-point drop on WebVoyager.
- [15]
Nemotron 3 Super was scored with NeMo Evaluator/NeMo Skills and Nemotron 3.5 with NeMo Gym, so the author treats the NVFP4-to-NVFP4 deltas as directional.
- [16]
On the card, the NVFP4 model posts SWE-bench Verified 71.07.
- [17]
The card's DGX Spark Quick Start uses vllm/vllm-openai:v0.32.0 (or the pinned nightly) with --max-num-seqs 1 and --gpu-memory-utilization 0.75.
- [18]
For vision on a Mac today, the author recommends llama.cpp with the mmproj file.
- [19]
A flat 4 bits per parameter across 120B parameters would be 60 GB.
- [20]
Going by Unsloth's numbers, llama.cpp should run the model with vision on a 128 GB Mac or a 96 GB workstation card.
- [21]
The 128 GB planning figure leaves 55.3 GB above the measured 72.7 GB MLX peak.
Get this ledger inside ChatGPT. Ask about any of our stories and the answer brings its figure, each claim’s verdict and the sources. Free, no account.
Sources
1 independent publisher whose own reporting we read for this story.
- dev.toHow to Run Nemotron 3.5 Super VL Locally: You Need 128GB, Not a Blackwell GPU
1 article · October 16, 2026
Topics and entities
Follow any of these and your For You feed starts watching them — no settings page required.