Loading today’s stories
Build1 distinct publisher3 min readPublished
A 15M-parameter model streams English text on a 2007 PSP at about one token per second. That is the extreme end of a sizing rule. The harder half of that rule is checking whether the file that fits is a format its own maintainer recommends.
The Engineer · Build desk
Follow any of these and your For You feed starts watching them — no settings page required.
build
Unsloth's 10% quant claim is really about which machines can run a 27B model1 distinct publisher
build
A 27B Apache-2.0 model in 17GB makes local inference a wiring decision, not a demo1 distinct publisher
build
Qwen 3.8's Apache-licensed 27B is the one you can actually own, and its KV cache is why1 distinct publisher
build
RamaLama ships models as OCI images you can inspect and sign1 distinct publisher
Compiled by The EngineerSomething wrong?How this is made
The byte budget is the only part of this stack that will not negotiate. Seventeen megabytes of int8 weights [2] against 64 MB of system RAM [1] is about 27 percent of the machine, leaving roughly 47 MB [1] for the runtime, the KV cache over a 256-token context, the logits array across a 32,000-token vocabulary [4], the framebuffer, and whatever the OS holds. Headroom is where generation actually happens. A model sized to fill the RAM loads and then dies mid-completion, which is the sequence the dev.to guide's author reports watching in developer threads: pick the trending checkpoint, wait 45 minutes for the download, discover the fit at OOM [6].
Reversed, the hardware produces the shortlist and the shortlist is short [7]. The author's process assumes llama.cpp [17], and he is explicit that he has not run a model on a PSP himself, only local GGUF models on his own servers for summarization and embedding [8]. At 64 tokens per minute [3] the PSP is running at roughly 1.1 tokens per second [2], so this is a demonstration of a budget rather than a device anyone will deploy.
The build did one thing worth copying on any target. They compiled the reference C runtime with matching floating-point flags and diffed the PSP output byte for byte [5]. Either the diff comes back clean or the engine is wrong somewhere; there is no third result. Quantized output that reads plausibly is not evidence of a correct matmul.
Fitting settles the loader, and the second test is whether the format you fit into is one its maintainer endorses for your task. Unsloth's UD-IQ1_S build of Qwen3.8-27B is 6.2 GB and 89 percent smaller than the original [12], which puts the unquantized weights near 56 GB [3] and is the entire reason anyone considers a 27B model on a desktop. It retains about 72 percent of top-1 accuracy [12]. Their docs then say accuracy drops sharply below the UD-Q2_K_XL format, that 1-bit models can produce empty responses in non-thinking mode, and that tool calls with heavily quantized models are not recommended because they fail, repeat, or never fire [13]. Read the labels: IQ1 is one bit and Q2 is two, so the file carrying the compression headline sits below the floor the same documentation draws [6]. The reported 10 percent top-1 advantage at equal file size comes from post-training quantization against a curated calibration dataset [11], so it transfers to the extent your prompt distribution resembles that dataset and their eval.
The 90M tool-caller in the Falcon-H1-Tiny family is the case where small wins outright: 94.44 percent relevance detection, matching a 270M Function Gemma on that task [9] with three times fewer parameters [5], at roughly 90 MB in Q8_0 on a phone or a Pi [10]. Relevance detection means knowing when a function should be called at all, so that number moves to your system only if your tool count, schema shapes, and share of no-call prompts look like theirs. The guide sources it to an r/LocalLLaMA discussion, not an independent eval [9].
GPT-3 was 175 billion parameters in 2020 [14], about 11,700 times the PSP model [4], and scale did not stop mattering; the useful conclusion is that the defensible shortlist is the intersection of files that load with headroom and formats the maintainer recommends for the job. For routing, constrained generation, and JSON extraction, that intersection has entries [18]. For a tool-calling agent at one bit, Unsloth's own documentation says it is empty [13]. The claim that a well-designed 90M architecture scaled to 1B would cover chat, tool calling, and light coding under 500 MB is a community projection, not a shipped result [16].
Ranked by verification strength, evidence, and original report placement.
A developer ran a working language model on a Sony PSP-2000, a 333 MHz MIPS handheld from 2007 with 64 MB of RAM, streaming English text to its LCD at one to two tokens per second.
The PSP model is Karpathy's stories15M, int8-quantized down to about 17 MB, running on roughly 1,100 lines of pure C.
A 64-token completion on the PSP build takes about a minute.
The PSP project writeup details a 15M-parameter TinyStories model, int8-quantized with group size 64 and symmetric scaling, 17 MB on disk, with a 32,000-token vocabulary and 256-token context, cross-compiled in pure C with pspdev in Docker.
The builders also compiled the reference C runtime with matching floating-point flags and diffed its output byte for byte against the PSP build; if the diff is not clean, the engine is wrong somewhere.
The author's recommended order is to start from the hardware you actually have, then find the biggest model that fits it with headroom.
Distinct publishers with included, body-backed reporting in this cluster.
1 article · September 6, 2026
Evidence-backed comparisons of source perspectives and observed adoption signals. Read the methodology
Which Builder, Operator, and Investor concerns the observed source mix emphasized—not a truth score.
Evidence, demonstrated adoption, hype gap, incentives, and confidence are assessed independently, each on its own current evidence. How these are measured.
Everything routes through one dev.to guide
The PSP port, the 94.44% relevance figure from a Reddit thread and Unsloth's accuracy tables all reach readers through a single writer who says up front that he never touched a PSP and that his quantization numbers come from published docs. The one piece of real verification in the story belongs to the PSP builders, who diffed their engine's output byte for byte against the reference C runtime. Nothing comparable stands behind any of the accuracy percentages.
A hobby port, a vendor release, one writer's servers
What is demonstrably running is small and mostly individual: 17 MB of weights on a 2007 handheld, Unsloth's tier table of quantized files, a 90M Falcon variant wired into VS Code through Continue, and the author's own Qwen GGUF doing triage on his machines. There are no download counts, no organisational deployments, and no third party reporting production use of any of it.
The prose is careful, but one row in the table overreaches
The writing is unusually self-limiting: it names Unsloth's figures as vendor benchmarks, repeats the Hacker News objection that 2-bit is a lobotomy for a model this capable, and quotes Unsloth's docs saying tool calls with heavily quantized files fail or never fire. Against that restraint sits the recommendation itself, where the 6.2 GB 1-bit file offered to 8 GB machines is a tier below the floor those same docs set. That single line in the tier list is where the gap actually sits, not in how the piece frames its claims.
The quantizer grading its own quants
The up-to-10% accuracy advantage comes from Unsloth's calibration set and Unsloth's tables, which is the ordinary state of quantization benchmarking and still worth naming when those tables are being used to choose a download. The Falcon parity number arrives via a community thread about a vendor's own release. The writer's stake is small and stated: he runs local GGUF models for his own summarization and embedding work and points readers at documentation rather than a product.
The method holds up, the percentages don't
The procedure holds regardless of who published which benchmark: check available memory under your real workload, budget 1.3 to 1.5 times the file size at moderate context, then choose a quant tier and keep small models on narrow tasks. The specific accuracy numbers attached to specific files do depend on one party's testing, and one writer relayed all of them, so the tier table is a starting point for your own measurement rather than a measurement.