Build1 distinct publisher3 min readPublished
Overhead is m over k and tolerance is m, which is all the arithmetic this choice needs. The trap is sizing m from a byte-level Reed-Solomon demo, where ten parity bytes only repair five errors at unknown positions.
The Engineer · Build desk

Compiled by The EngineerSomething wrong?How this is made
Ten parity bytes recover five corrupted bytes, and that halving is the part worth carrying into a design review. RSCodec(10) has to locate the damage before it can correct it, so the tutorial's own budget line is 10 // 2 = 5 [8]. A storage tier never pays for location. A dead disk announces itself, the fragment is absent rather than wrong, and the position comes free. So RS 6-3 keeps three parity fragments and survives three simultaneous losses [12][2], where the byte-level demo keeps ten and survives five [8][8]. Size m from the code sample and you are out by a factor of two, in whichever direction is more expensive.
Two lines carry the rest of the decision. Overhead is m/k, and tolerance is m [2][3]. HDFS's default of three copies per block tolerates two losses [1][3]. RS 6-3 tolerates three and pays 50% [1][2]. RS 6-2 tolerates the same two as replication and pays 33%, a sixth of replication's storage penalty [5]. The article promises to tell you when RAID 6 fits, when HDFS with erasure coding fits, and when simple replication fits [11]. The excerpt never reaches the criteria. Overhead, tolerance and repair reads are the criteria.
The word doing the work in "any k" is any, and it comes from the matrix. Encoding multiplies the k data fragments by a Vandermonde or Cauchy generator matrix, both chosen because any subset of their rows is invertible [5]. Decoding assembles a square submatrix from whatever survived, inverts it in the finite field, and multiplies the missing fragments back [6]. Because every k-subset inverts, the code is indifferent to which fragments died, and to whether the thing that died was a disk, a node or a whole rack [3]. A mirrored RAID is not indifferent, since it protects one specific disk with one specific partner [14].
The demo library is pure Python [7]. That is the right call for making field arithmetic visible on a laptop, and I would not put a pure-Python Galois multiply anywhere near a write path. The runnable snippet also flips bits inside a message, while the objective it illustrates is losing whole fragments and rebuilding with no identical copy anywhere [16]. Those are two different failure models, and the second is the one your rack produces.
For the 50% to transfer, RS 6-3's nine fragments need nine places that fail independently. Spread them across five hosts and at least one host is holding two of the nine, so one host failure consumes two thirds of the three-loss budget [7]. The code is then protecting against fragment loss while the hardware fails in host-sized units, and no amount of finite-field algebra reconciles those two units for you.
Ranked by verification strength, evidence, and original report placement.
HDFS by default uses a replication factor of 3, with each block stored on three different machines, giving 200% extra storage cost: for every useful terabyte, two additional terabytes are stored purely as backup.
Erasure coding divides data into k data fragments and computes m additional parity fragments using algebra over finite (Galois) fields, producing k+m fragments in total, of which any k are enough to reconstruct the complete original file.
It does not matter which m fragments are missing, nor whether disks, nodes or entire racks fail; as long as k of the k+m fragments survive, reconstruction is mathematically possible.
Reed-Solomon, published by Irving Reed and Gustave Solomon in 1960, treats each data fragment as a point on a polynomial over a finite field, typically GF(256) because it fits exactly into a byte; k points define a polynomial of degree k-1, and the m parity fragments are that same polynomial evaluated at m additional points.
Encoding multiplies the k data fragments by a generator matrix (Vandermonde or Cauchy, both of which guarantee that any subset of rows is invertible) to produce the m parity fragments.
Decoding does the inverse: from the surviving fragments it assembles a square submatrix, inverts it in the finite field, and multiplies to recover the lost fragments.
Distinct publishers with included, body-backed reporting in this cluster.
dev.to
1 article · August 30, 2026
Follow any of these and your For You feed starts watching them — no settings page required.
build
Force the tool call, then hand Lightsail a long-lived key1 distinct publisher
build
A cache hit is a quota refund: semantic caching with trigrams and no vector database1 distinct publisher
build
Stop timing your GraphQL tests and start counting loader calls1 distinct publisher
build
Agent memory rots by accumulation, and the missing primitive is a supersession key1 distinct publisher
Evidence-backed comparisons of source perspectives and observed adoption signals. Read the methodology
Which Builder, Operator, and Investor concerns the observed source mix emphasized—not a truth score.
Evidence, demonstrated adoption, hype gap, incentives, and confidence are assessed independently, each on its own current evidence. How these are measured.
Checkable maths, unfinished demo
What carries the headline is arithmetic a reader can verify without leaving the page: three parity fragments over six data fragments is 50% overhead, any six of nine rebuild the file, and Reed-Solomon over GF(256) has been public since 1960. That is why a single unreviewed tutorial still scores respectably. What drags it down is everything the text asserts but does not show — the fragment-loss reconstruction it promises never runs, the diagram disagrees with the scheme it illustrates, and the instructions end mid-sentence.
No adoption signal
The only usage evidence is one sentence saying HDFS, S3-style object stores and RAID 6 do this every day — no version, no cluster, no date, no operator saying what it cost them. Nothing here counts as an observed deployment, and we will not manufacture one from a tutorial's opening paragraph.
Cost win real, caveats thin
The 200%-to-50% framing in the title holds on disk footprint, so this is not inflation — it is omission. Storage is the one axis on which erasure coding wins cleanly; repair reads, fragment placement across too-few hosts, and encode/decode CPU are the axes on which it charges you back, and none of them appear. Add that the demo a reader runs proves a weaker property than the one being sold, and the piece reads a notch more settled than its evidence.
Little to sell
No vendor sits behind this. The library is a free pure-Python package, the systems named are open or generic, and there is no pricing, sponsorship or product on offer. What remains is the ordinary pull of developer-platform publishing: a round number in the headline and a six-item objective list travel further than a caveat about repair traffic, which is the direction the omissions happen to lean.
Trust the maths, not the details
We would defend the core numbers to any operator: they are derivable, and their derivation is on the page. We would not lean on this story for anything procedural, because a piece that mislabels its own diagram and never delivers its headline experiment has told us how carefully it was checked — and with one publisher, nobody else caught it either.