Loading today’s stories
Build1 distinct publisher3 min readPublished
One engineer counted the checks in his own repositories that assert a conclusion: 204 of them, 22 that can be shown to fail. The remaining 182 have never met a known-bad input.
The Engineer · Build desk
Compiled by The EngineerSomething wrong?How this is made
Fifty-four control probes spread over 22 guards averages about 2.4 each [3], and that is the part the headline ratio hides. The guards that got a negative control usually got more than one; 182 got nothing [1]. There are more than three unprobed guards in that tree for every probe that exists in it [4]. So this is not thin coverage spread evenly. It is a question the author asked hard about a few checks and never asked about the rest.
The reason the gap is not merely a coverage statistic: without a bad input, a check that cannot find anything and a check with nothing to find emit the same result, and the pipeline page only shows you the result. The assertions in scope are claims like no workflow downloading its cache over the network, or a feature flag matching the deployed spec [c5x]. Nobody eyeballs those. That is why they were automated in the first place.
The failure in the deploy pipeline at least announced itself. The other two, from the same week of production tooling [11], returned confident wrong answers. An autonomous harvester wrote seven usable records, hit a non-fatal warning, exited non-zero, and booked its own finished work as failed and retryable, because interrupted-with-partial-results had no representation at all [9]. An error classifier looking for the pattern 50[024] anywhere in output matched the 500 inside "4258 of 5000 quota points remaining" and filed a successful run as a server failure [10]. Every field that classifier read was real; it was answering a different question from the one asked [10]. The author's diagnosis across all three cases is that the check watched a messenger while the artifact told a different story [12].
Read the ratio for what it is. Eleven percent covered, 89 percent never handed a bad input [4], is a self-audit of code its own author wrote, published on dev.to under the handle heinrichneb [17], and it establishes nothing about anyone else's tooling. What transfers is the cheapness: the convention is a marker string in a test name [c6b], and the definition is narrow enough to grep for, so anyone can produce their own number this week and find out whether 11 percent is unusual or ordinary.
What gives the count weight is the job it describes. The author used to produce artifacts and verify a few of them; an agent now produces most of the artifacts, which makes those guards the codebase his judgment actually ships through [13]. He set out to disagree with Michael Amachree's line that AI made him a worse reviewer rather than a worse coder, and ended up sharpening it instead: the model is not the weak link, the unfalsifiable green check is [14][16]. The rule he now applies is to judge the artifact rather than the messenger, and to treat any check that only ever reads messengers as unverified regardless of its colour [15].
Ranked by verification strength, evidence, and original report placement.
An audit counted 204 conclusion-bearing guard files across three of the author's own repositories.
Of those 204 guards, 22 have at least one negative control probe.
There are 54 control probes in total across the three repositories.
The author states the coverage is 11 percent, and that the other 89 percent have never once been shown a known-bad input.
A conclusion-bearing guard is defined as any test that reads source code, config or system state and asserts a claim about it: the tests that stand in for a human reviewer, not tests of whether a function returns 4.
Examples given of such assertions include "no workflow downloads its cache over the network", "every page passes the same quarter filter", and "this feature flag matches the deployed spec".
Follow any of these and your For You feed starts watching them — no settings page required.
Evidence-backed comparisons of source perspectives and observed adoption signals. Read the methodology
Which Builder, Operator, and Investor concerns the observed source mix emphasized—not a truth score.
Evidence, demonstrated adoption, hype gap, incentives, and confidence are assessed independently, each on its own current evidence. How these are measured.
Concrete but single-source and self-audited
The numbers are specific, internally consistent and reproducible in principle - the post publishes the counting script and the marker convention - and the three failure cases are described with enough mechanism (exit 127, non-zero exit on non-fatal warning, 50[024] substring match) to be checked against similar systems. But every figure comes from one publisher, the auditor is the author of the guards, and the counter is explicitly a marker-based proxy accurate to plus or minus a few points. No independent replication, no second observer, no external dataset.
One engineer's repositories, no external uptake
Adoption evidence is confined to the author's own environment: a guard suite across three of his repositories, a private KONTROLLE: marker convention, and three production incidents in his own tooling within seven days. The post invites others to run the script and report numbers above 30 percent, which is an explicit acknowledgement that no such external data exists yet. No third party, team, vendor or project is shown using the practice.
Modestly overstated in scope, not in substance
The mechanism claims are well matched to the evidence and the post volunteers its own limits, including the plus-or-minus-a-few-points caveat and an invitation to be contradicted. The overstatement is one of scope: a headline asserting AI promoted every developer to reviewer and that nobody tested the reviewer generalises from one engineer's 204 guards and three incidents to the whole industry, with no cross-team measurement offered. Directionally the argument is stronger than its evidence base is wide.
Attention and convention promotion, no commercial stake
The visible incentives are authorial: a developer-platform post responding to another popular post, ending with an explicit call for comments and for readers reporting coverage above 30 percent. It promotes the author's own vocabulary (conclusion-bearing guard, KONTROLLE: markers) and his script. No product, employer, sponsor, vendor or funding relationship is disclosed in the supplied source, so there is no evidence of a commercial incentive - only engagement and reputational ones.
Moderate: coherent and self-limiting, but unreplicated
Confidence is limited by cluster structure more than by the writing. One publisher, one self-auditing author, a proxy counter, and derived figures that are only as good as the inputs. Against that, the definitions are operational, the arithmetic is consistent, the incidents carry checkable mechanisms, and the author flags his own error bars - so the qualitative conclusion (unfalsifiable checks are indistinguishable from working checks) is more reliable than the specific percentages.
build
A GAN beauty filter is a device budget allocation, not a feature toggle1 distinct publisher
build
Every viewer hits your HLS key endpoint in the same second, and almost nobody tests it1 distinct publisher
build
912MB to 108MB is mostly typing now, and that weakens the base-image excuse in review1 distinct publisher
build
The optional EntityManager is the bug: moving the transaction boundary into AsyncLocalStorage1 distinct publisher
Distinct publishers with included, body-backed reporting in this cluster.
dev.to
1 article · August 24, 2026