Loading today’s stories
Build1 distinct publisher3 min readUpdated
A 28.9-million-parameter model now runs on an ESP32-S3 that was topping out at 260K two years ago. The gain came from moving most of the weights into flash and reading 450 bytes per token.
The Engineer · Build desk
Compiled by The EngineerSomething wrong?How this is made
On July 25, 2026, a repository called esp32-ai reached the front page of Hacker News running a 28.9-million-parameter language model entirely on an ESP32-S3, a microcontroller that costs about eight dollars and carries 512KB of SRAM, 8MB of PSRAM and 16MB of flash [1]. The previous record for a language model on that chip family was about 260 thousand parameters [2], so the question worth asking is not whether it works but what changed, and the answer is where the weights sit.
The arithmetic that used to end this conversation still holds. Quantized to 4 bits, 28.9M parameters need roughly 14.9MB [3], which fits in the 16MB of flash with around 1.1MB spare [4] but is far too slow to stream through on every token [3]. The developer, who goes by slvDev, borrowed per-layer embeddings from Google's Gemma 3n [5]. Embeddings are lookups, not matrix math, and most of a small model's parameters live in them [6]. So the model splits: a dense reasoning core of roughly 560 thousand parameters, about 2 percent of the total [12], runs in RAM every step, while 25 million parameters, roughly 86 percent of the model [13], sit in flash as a memory-mapped table [7].
Generating a token touches about six rows of that table, roughly 450 bytes of flash reads [8]. At the reported 9.88 tokens per second [1] that is about 4.4KB per second of flash traffic [14]. SRAM holds activations and normalization weights, PSRAM holds the core and the output head, flash holds the embedding table [9]. The processor is the same processor. The hierarchy is new.
The lineage is worth naming, because it shows the gain is cumulative engineering rather than a jump in parts. Most microcontroller LLM ports descend from Andrej Karpathy's llama2.c and from DaveBben's esp32-llm, which in 2024 ran a 260K model at 19 tokens per second on the same chip family [10]. Two years later the parameter count is about 111 times higher [11] on hardware that costs the same [1], at a lower token rate. That trade, throughput for capacity, is the whole story.
Sort the field honestly and it splits into tiers where weights and compute both live on the chip, versus where the chip only does the math [15]. The most interesting near-miss belongs to the second category. On August 5, 2026, a developer going by cyfrit posted p-for-llm, a 180.9-million-parameter mixture-of-experts model on an ESP32-P4 with 12 layers, 29 experts per layer, top-1 routing, ternary BitNet-style quantization and a vocabulary pruned from Qwen, on a board costing six to ten dollars [16]. It runs at roughly 9 tokens per second, follows simple ChatML instructions, and has a tool-calling demo the author himself describes as frequently going off the rails [17]. The asterisk: about 44MiB of weights are pushed over USB at startup, with SD-card autonomy still future work [18]. That is more than the 24MB of combined flash and PSRAM on the S3 [19], which is exactly why it needs the umbilical. It was trained on about 12 billion tokens on a single consumer GPU [20], and it got two points on Hacker News [16].
Watch whether p-for-llm's SD-card path lands, since that is the line between a demo with a host attached and a device. Watch also whether per-layer embeddings show up in other ports, because an architectural trick that anyone can copy sets a new floor rather than a record.
Ranked by verification strength, evidence, and original report placement.
On July 25, 2026, a repository called esp32-ai hit the front page of Hacker News with a 28.9-million-parameter language model running entirely on an ESP32-S3 microcontroller that costs about eight dollars and has 512KB of SRAM, 8MB of PSRAM and 16MB of flash, generating text at 9.88 tokens per second, roughly reading speed. Tom's Hardware, The Register, Adafruit and CNX Software covered it within a week.
SRAM holds activations and normalization weights, PSRAM holds the core and the output head, and flash holds the embedding table; the breakthrough is a memory hierarchy, not a faster processor.
As of August 2026 the source classifies projects in this space into tiers: Tier 1, true on-device, where weights and computation both live on the microcontroller, with esp32-ai as the flagship example; Tier 2, on-device compute with weights streamed from a host, such as p-for-llm's USB loading or a wifi-llm experiment streaming a 16MB model layer-by-layer into 300KB of usable RAM; Tier 3, distributed microcontrollers passing activations over radio.
The previous record for a language model on an ESP32 was about 260 thousand parameters.
A 28.9M-parameter model quantized to 4 bits needs about 14.9MB; it fits in flash, but flash is far too slow to stream weights through on every token.
The developer, who goes by slvDev, borrowed the idea of per-layer embeddings from Google's Gemma 3n.
Distinct publishers with included, body-backed reporting in this cluster.
dev.to
1 article · August 17, 2026
Evidence-backed comparisons of source perspectives and observed adoption signals. Read the methodology
Which Builder, Operator, and Investor concerns the observed source mix emphasized—not a truth score.
Evidence, demonstrated adoption, hype gap, incentives, and confidence are assessed independently, each on its own current evidence. How these are measured.
Detailed and internally consistent, but single-publisher and self-reported
The technical account is unusually specific and checkable in principle — storage tiers, 4-bit footprint of 14.9MB against 16MB flash, a ~560K-parameter RAM core versus 25M flash-resident parameters, ~450 bytes read per token, and a named architectural antecedent in Gemma 3n. The arithmetic in the derived claims is consistent with the stated figures. But the cluster contains exactly one source, a community blog post, and every throughput and parameter number originates with the project authors; the press coverage it cites is not present for corroboration, and there are no independent benchmarks, power measurements, or evals.
Hobbyist repositories and one viral post; no production use shown
Observed adoption is attention plus artifacts: one repository on the Hacker News front page with same-week trade coverage, a second and more capable project that drew two points and vanished, and a three-board distributed experiment at about thirty coherent words. Nothing in the cluster shows a shipped product, a commercial deployment, downstream users, or an integration by Espressif — whose own official LLM offering is cloud inference and whose newest chip adds SIMD and radios rather than neural hardware. Adoption is therefore real but confined to the enthusiast tier.
Ecosystem framing runs ahead of capability; this source partly corrects it
The surrounding narrative — 'an LLM on an $8 chip', a viral comparison to a quarter of GPT-1 by parameter count — overstates what the artifacts do: the flagship tier-1 model is TinyStories-trained and cannot answer questions or follow instructions, the most capable project needs a 44MiB USB load at every boot, and the distributed cluster has produced about thirty coherent words. The gap is positive but modest rather than large, because this source itself does the deflating: it labels tiers, kills the NPU myth, quotes the authors' own caveats, and says plainly that parameters are not capability.
Attention incentives on both author and projects; no disclosed commercial stake
The visible incentives are reputational rather than financial: independent developers publishing on Hacker News benefit from framing that emphasizes the $8 price and the 100x parameter jump, and the piece runs on a community developer platform where a striking premise carries reach. Offsetting this, the author is not shown to be affiliated with any of the projects or with Espressif, and repeatedly argues against the flattering reading. Espressif has an evident commercial interest in the client-side framing given its cloud LLM offering and NPU-free roadmap, but the cluster supplies no funding, sponsorship, or vendor-relationship facts, so the score reflects only these observable pressures.
Moderate: coherent technical account, one publisher, unverified numbers
Confidence is limited primarily by source concentration. All fourteen canonical claims trace to a single dev.to post, the derived claims are arithmetic on that post's figures, and the referenced independent coverage is absent from the cluster. The account's internal consistency, named antecedents, and voluntary disclosure of limitations raise confidence above a bare single-source floor, but throughput, capability, and power characteristics remain unreplicated here.
Follow any of these and your For You feed starts watching them — no settings page required.
build
A 180M-parameter model on a $6 board says the microcontroller limit was active params, not total1 distinct publisher
invest
Etched's $10.3B mark prices a non-Nvidia inference bet at ten times booked orders1 distinct publisher
build
A 27B Apache-2.0 model in 17GB makes local inference a wiring decision, not a demo1 distinct publisher
invest
Tiny corp wants Etched's numbers. Jane Street led $700M at $21B without publishing any1 distinct publisher