Build1 distinct publisher3 min readPublished
A capacity model for streaming a mixture-of-experts checkpoint agreed with a stranger's conversion to within 1.1%, but that agreement was two errors of opposite sign, and the resolved answer was sitting in GGUF metadata on the author's own disk.
The Engineer · Build desk

Compiled by The EngineerSomething wrong?How this is made
Both halves of the cancellation are checkable. The wrong layer count added 256 experts at 14,155,776 bytes each, which is 3,623,878,656 bytes, or 3.62 GB of routed experts [13][1]. Net of the dense layer that extra mixture-of-experts layer displaced, the inflation was +3.54 GB, so the thing it displaced was worth roughly 80 MB [13][2]. Against that sat -5.27 GB of parts the model never attempted: the vision and audio encoders, the multi-token-prediction module, the RMS norms, and the published conversion's choice to keep non-expert tensors at 8 bits rather than 4 [14]. The sum lands 1.73 GB low, reported as 1.1% against a 2% tolerance [15][3].
Back the container out of those two pairs. A 1.73 GB miss at 1.1% implies a published file near 157 GB; a 5.27 GB miss at 3.4% implies near 155 GB [4]. The coverage gap was always about 5 GB on a roughly 155 GB artifact, and the layer-count error was hiding most of it [14][3].
What failed here lives in interpretation, not arithmetic: a config file describes an architecture in the vocabulary of the framework that trained it, so reading one means inferring what a field name meant to its author [20]. `dense_mlp_idx=2` supports "the layer at index 2 is dense" and "layers below index 2 are dense", which give 41 and 40 mixture-of-experts layers [5]. The GGUF header is downstream of that question: the conversion script writes its key/value metadata after it has already resolved every ambiguity of this kind [7]. Forty-two blocks, two of them dense, leaves forty routed layers, which is the second reading [9][8].
The original limitations section addressed this directly. It bounded the effect on bytes per token at 2.4% and called it immaterial to the conclusion [4]. That bound held for the quantity it described, but it said nothing about the validation residual, which moved 2.3 percentage points and crossed a tolerance set at 2% [7][6] - roughly a threefold increase in reported error [6]. For 1.1% to have meant what it looked like, the uncounted parts needed to net to zero and the layer count needed to be right; instead, both were wrong [14][8].
Gate 1 is the part worth copying. It compares computed integers against a published layout with no tolerance at all, and 256 experts at a stride of 1,769,472 bytes is exactly the 452,984,832-byte per-layer blob it checks against [17][5]. Equality has no room inside it for two errors to meet, which is the argument for preferring an exact gate wherever the domain publishes exact integers [16][18]. The load-bearing output here, bytes streamed per token, depends on that stride, and the author reports it improved rather than degraded [19].
Reading the metadata took about forty lines of Python, and the parser was wrong the first time in a way that generalises [9]. GGUF stores string arrays length-prefixed; consuming only the first few elements shifted every subsequent offset, and the parse died later at an invalid-looking type code rather than at the array that caused it [10]. The format reports the failure in the least useful place available. Consume every element even when you only intend to print four [10].
Ranked by verification strength, evidence, and original report placement.
The author wrote a cost model estimating bytes streamed per token for a mixture-of-experts model streamed off an SSD, and the model gates itself twice against artifacts the author had no part in producing.
Gate 1 reproduced a third-party runtime's on-disk container byte-for-byte.
Gate 2 predicted the size of a model conversion published by different people using a different tool, and came within 1.1% against a 2% tolerance.
The original post's limitations section flagged that dense_mlp_idx=2 was ambiguous, stated that the post took the one-dense-layer reading, that the choice moved bytes/token by 2.4%, and that it was immaterial to the conclusion and unresolved regardless.
dense_mlp_idx=2 could reasonably mean 'the layer at index 2 is dense' or 'layers below index 2 are dense', which give 41 and 40 mixture-of-experts layers respectively.
Correcting the assumption took Gate 2 from 1.1% off to 3.4% off, which is outside the tolerance it had been passing.
Distinct publishers with included, body-backed reporting in this cluster.
dev.to
1 article · August 30, 2026
Follow any of these and your For You feed starts watching them — no settings page required.
build
A 27B Apache-2.0 model in 17GB makes local inference a wiring decision, not a demo1 distinct publisher
build
Force the tool call, then hand Lightsail a long-lived key1 distinct publisher
build
A cache hit is a quota refund: semantic caching with trigrams and no vector database1 distinct publisher
build
Stop timing your GraphQL tests and start counting loader calls1 distinct publisher
Evidence-backed comparisons of source perspectives and observed adoption signals. Read the methodology
Which Builder, Operator, and Investor concerns the observed source mix emphasized—not a truth score.
Evidence, demonstrated adoption, hype gap, incentives, and confidence are assessed independently, each on its own current evidence. How these are measured.
Arithmetic you can check, from one pair of hands
The internal bookkeeping holds up under inspection: 256 experts at 14,155,776 bytes really is 3.62 GB, the per-layer blob of 452,984,832 really is 256 times the 1,769,472-byte stride, and the +3.54 GB net implies the displaced dense layer was worth about 80 MB. What is missing is anyone else. The runtime behind Gate 1 and the conversion behind Gate 2 are never named, the conversion's size survives only as a percentage denominator, and both corroborating checks on the corrected figure are the author's own.
One machine, one operator
There is nothing to measure yet. The only usage on record is the author running the conversion locally and reading page-in counters off the kernel, plus a companion script of his own that agrees to 0.2%. No third party has adopted the model, repeated either gate, or reported a number of their own.
Understated, via the unusual route of self-demotion
The headline finding is a failure the author volunteered against his own earlier work, and the sharpest conclusion — that a tolerance never showed the two sides agreed for the same reason — costs him the result he was proudest of. The one place the framing leans forward is 'the number got better': that rests on a 0.2% agreement between two of his own scripts and a measured bracket so broad it would have contained the pre-correction figure too.
Nothing sold, and the story bills its own author
No vendor, funding round, license, or product sits behind this; the artifacts checked belong to strangers and the checker gains nothing if they match. Publishing on dev.to buys reputation, and a well-told mea culpa buys a particular kind of it — but the coin spent is the author's own prior claim of a 1.1% match, which is an odd way to inflate anything.
Firm on the lesson, single-sourced on the numbers
Two things pull in opposite directions. The reasoning is unusually verifiable for a single-source story — the disclosed 2.4% bound on bytes per token turns out to describe the move from 3.482 to 3.397 GB almost exactly, so the author's earlier uncertainty estimate was sound even though his reading of the field was not. Against that, one narrator supplies every measurement, no external identifier exists for the artifacts on either side of the gates, and the timing of the fail-state check is unverifiable. The general point about cancelling errors does not depend on any of that; the specific gigabyte figures do.