BuildNot yet confirmed elsewhere1 publisher2 min readPublished Updated
VIDRAFT's 4-bit Darwin-180B build streams most of its weights from SSD on a 40 GB laptop
VIDRAFT says POCKET-Darwin-180B, a 111 GB 4-bit GGUF build of its 180B mixture-of-experts model, runs in llama.cpp on about $1,400 of consumer hardware. The accuracy evidence so far is one MMLU-Pro comparison that VIDRAFT reports itself.
The Engineer · Build desk
What happened
- Each token is routed through 10 of the model's 512 experts, so only about 3 billion parameters are active per token, according to a dev.to write-up of the release.
- Experts not held in memory are streamed from SSD on demand through memory mapping, and the write-up names storage I/O bandwidth as the bottleneck.
- The stated targets are a laptop with as little as 8 GB of VRAM and 32 GB of system RAM, or a CPU-only mini PC with about 128 GB of unified RAM.
- The BF16 original took 360 GB across 131 files, and VIDRAFT credits a method it calls graft quantization for the four-file build.
Why it matters
- decision Anyone pricing the laptop build has to treat sustained SSD read bandwidth as the spec that sets tokens per second, ahead of the choice of CPU.
- constraint A speed result from the 128 GB mini PC will not predict the laptop, because one holds the whole file in memory and the other pages most of it from disk.
- cost Task-level accuracy checks fall to each adopter, since parity with the BF16 original has been shown on one aggregate benchmark only.
- capability If the claims hold, a stock llama.cpp build is enough to evaluate a 180B-class model on a single local machine with no custom tooling.
Ten experts out of 512 is about 2% of the pool on any one token [13]. That small slice is why SSD-backed inference can work at all, and the slice moves from token to token. If the active weights are stored at the file's average density, each token touches roughly 1.85 GB of them [14].
The laptop setup has 40 GB of fast memory. That leaves 71 GB of the 111 GB file, about 64%, on the drive [15]. Assume the router spreads load evenly across experts and the page cache holds nothing beyond what fits. Then about 1.2 GB of each token's weights comes off the SSD [16], and every gigabyte per second of sustained read buys a little under one token per second [17]. Real throughput beats that floor only as far as the router reuses the same experts on consecutive tokens.
The 128 GB mini PC is a different case. The whole file fits with about 17 GB to spare [18], so once the pages are warm nothing streams from disk and memory bandwidth sets the pace. A dev.to write-up of the release prices a "consumer setup" at about $1,400 [4] without saying which of the two machines that figure describes.
The file size also says something about the format. A flat 4 bits across 180 billion weights would come to 90 GB, and 111 GB works out to about 4.9 bits per weight [19]. Either some tensors are held above 4 bits or the format carries scale overhead. VIDRAFT reports zero accuracy loss on public benchmarks, according to the post [21]. Its author guesses at a targeted or layer-aware scheme and notes that standard Q4 quantization usually costs measurable quality [21]. For the MMLU-Pro parity [22] to transfer to another workload, that workload has to tolerate quantization error as well as one aggregate benchmark score does.
In my view the design is good engineering for the constraint it targets. Sparse routing plus memory mapping lets a file nearly three times the laptop's fast memory run at all [15]. The 250-to-1 cost ratio against a roughly $350,000 deployment of four to eight enterprise GPUs [4][20] sets a machine bought for capacity against one bought for speed. VIDRAFT positions the build for local development and evaluation [12]. To get the files, the post tells readers to search Hugging Face for VIDRAFT/POCKET-Darwin-180B [10]. For a release about leaving the data center, it also points readers to n1n.ai, a hosted OpenAI-compatible gateway, for API access [11].
What to watch
- Independent llama.cpp runs on the 8 GB VRAM, 32 GB RAM laptop configuration that report tokens per second and name the SSD used.
- Task-level comparisons of the 4-bit build against the BF16 original beyond MMLU-Pro, on coding, long-context or tool-use work.
- A published Hugging Face repository for the GGUF files, whose per-tensor quantization types would show what graft quantization does.
Clarity's read
What the record supports and how the coverage leans. The claims behind it follow.
Reality
- Evidence30
- Adoption
- Insufficient
- Hype gap+45
- Incentives60
- Confidence35
Claim ledger
Ranked by verification strength, evidence, and original report placement.
- [1]
VIDRAFT has released POCKET-Darwin-180B, a 4-bit GGUF-quantized, llama.cpp-compatible build of its Darwin-180B-RSI model, a model in the 180-billion-parameter range, which it says runs on consumer hardware including CPU-only laptops and mini PCs.
- [2]
The compressed build is 111 GB across 4 GGUF files, down from 360 GB across 131 files in BF16; VIDRAFT credits 'graft quantization' and sparse MoE routing.
- [3]
Target hardware is consumer laptops with as little as 8 GB VRAM plus 32 GB system RAM, or CPU-only mini PCs with about 128 GB unified RAM.
- [4]
Estimated hardware cost is about $1,400 for a consumer setup, compared with about $350,000 for the 4-8x enterprise GPU alternative.
- [5]
About 3 billion parameters are active per token, not the full 180 billion.
- [6]
The model uses a mixture-of-experts design with 512 routed experts, of which 10 are activated for any single generated token, giving about 3 billion active parameters per token.
- [7]
Expert weights not in use can be streamed from SSD storage on demand via memory mapping without being held resident in VRAM or RAM; the bottleneck becomes storage I/O bandwidth rather than total VRAM capacity.
- [8]
The post advises engineers to independently verify task-specific performance on their own evaluation datasets before drawing production-readiness conclusions from a single aggregate benchmark.
- [9]
Standard llama.cpp usage applies; no custom tooling beyond a llama.cpp build is required.
- [10]
The post does not publish Hugging Face repository paths, GitHub links or download URLs, and tells readers to search Hugging Face for VIDRAFT/POCKET-Darwin-180B.
- [11]
For API access the post references n1n.ai as an OpenAI-compatible gateway offering access to Darwin-class models.
- [12]
POCKET-Darwin-180B is positioned for local development and evaluation use cases.
- [13]
10 of 512 experts is about 2% of the expert pool per token.
- [14]
At the file's average storage density, the roughly 3 billion active parameters per token amount to about 1.85 GB of weights touched per token.
- [15]
The laptop configuration has 40 GB of fast memory, leaving 71 GB (about 64%) of the 111 GB file outside memory; the file is about 2.8 times the laptop's fast memory.
- [16]
If routing spreads evenly and only what fits in memory is cached, about 1.2 GB of each token's weights is read from the SSD on the laptop configuration.
- [17]
Under the same even-routing assumption, each 1 GB/s of sustained SSD read supports a little under one token per second on the laptop configuration.
- [18]
On the 128 GB mini PC the full 111 GB file fits in memory with about 17 GB to spare.
- [19]
A flat 4-bit encoding of 180 billion weights would be 90 GB; the 111 GB file averages about 4.9 bits per weight.
- [20]
The enterprise GPU alternative costs about 250 times the consumer setup.
- [21]
VIDRAFT's 'graft quantization' is a 4-bit approach for which the team reports zero accuracy loss on public benchmark suites; the post says standard INT4/Q4 quantization typically introduces measurable degradation and suggests graft is a targeted or layer-aware strategy, with internal details not publicly disclosed.
ReportedInsufficientSource: dev.to write-up of the VIDRAFT release2 sources— create a free account to open themView cited source - [22]
The headline accuracy claim is identical MMLU-Pro scores between the BF16 original and the 4-bit compressed variant.
ReportedInsufficientSource: VIDRAFT, as relayed by dev.to write-up2 sources— create a free account to open themView cited source
Sources
1 independent publisher whose own reporting we read for this story.
- dev.toRunning a 180B-Parameter LLM on a Laptop Without a GPU: VIDRAFT's POCKET-Darwin-180B
1 article · October 2, 2026
Topics and entities
Follow any of these and your For You feed starts watching them — no settings page required.