Edge0's 35B would need 22 GB/s of SSD bandwidth without the file cache

Three billion active parameters at 4 bits is about 1.5 GB of expert weights per token, by the dev.to write-up's own calculation, which puts roughly half the 2.9 GB peak in the routed experts and the rest in shared weights and cache ^[12]. Multiply 1.5 GB by the reported decode rate and a cold read of the full active set on every token would need 22.4 to 26.6 GB/s ^[14]. No consumer SSD sustains that, as the same write-up notes ^[13]. So most expert reads are being served from the operating system's file cache, and dev.to treats "peak active memory" as the process resident set, with the cached memory-mapped pages sitting outside it ^[15].

On the benchmark machine there was room for that. Subtract the 2.9 GB resident set from 24 GB of unified memory and 21.1 GB is free, above the 19.6 GB the model card gives for the 4-bit 35B checkpoint ^[16]. A 24 GB Mac is an odd place to show that a 35B model does not need much RAM. After warmup the checkpoint can sit in the file cache, and the drive only has to serve what the cache evicts ^[15].

The prefetch design is careful engineering. Edge0 trains a small prerouter network to predict which experts the next step will need, so the runtime can start loading them while the current computation is still running, and frequently selected experts stay cached while less-used weights stay on disk ^[17]. Edge0 also ships Recover-LoRA, adapters trained to recover quality lost when the base models are cut to 4-bit precision ^[18]. Those evaluation averages were produced by Edge0 rather than an independent evaluator ^[19], and the repository calls tool use, multi-step planning and long-horizon autonomy weak ^[29].

For the 2.9 GB figure to transfer, the target machine needs enough free memory to keep most of the checkpoint cached. dev.to says a Mac with less memory cannot, and flags the whole reading as unchecked against the code ^[15]. There is no phone build to test it on either way. According to runtimewire, Zeng says the demo used 1 to 2.5 GB of peak memory, with no cloud connection and no remote GPU ^[23]. The repo's only chip-level mention of an iPhone is a troubleshooting entry: garbled, mixed-language output on Apple A18 and A18 Pro, traced to an older mlx ^[24]. The reposts that circulated an "iPhone 15 Pro and newer" floor got it from somewhere other than the README or the model cards ^[22].

Storage-backed inference has prior art. Apple's "LLM in a Flash" research described loading parameters from flash on demand to run models larger than available DRAM, and open-source projects including Swiftlet have shipped Swift and Metal implementations that stream sparse experts from an iPhone's storage, according to runtimewire ^[27]. What you can install today is a Python package that wants Python 3.10 or newer on an Apple Silicon Mac ^[25]. Budget the 19.6 GB on the drive you are willing to have read on every token, and note that the project does not say what months of that duty cycle do to a drive ^[26].