Build1 distinct publisher3 min readPublished
Unit tests prove a function behaves when something calls it, and nothing else. Proving that production still calls it needs a check that discovers the handlers independently of the dispatch list it audits.
The Engineer · Build desk

Compiled by The EngineerSomething wrong?How this is made
The load-bearing part of that wiring test is the loop over `pkgutil.iter_modules`. Discovery walks the gates package on disk, imports every module whose name ends in `_gate`, and collects every attribute that starts with `check_` and ends with `_gate` [8]. Only then does it compare that set against the names in `_MACHINE_GATE_CHECKS` and assert the difference is empty [9]. The docstring explains why the walk exists instead of a simpler import: build the expected list from the pipeline's own dispatch table and the comparison is meaningless, so it has to come from an independent source [10]. A test that derives its expectation from the thing under test cannot catch a divergence, because it is only ever checking the thing against itself.
The arithmetic on the demonstration is what makes it land. Cut the fifth gate from the tuple and the gate suite still reports 18 passed [6], while the wiring file fails on that same commit and names the gate [7]. One of five machine checks has gone silent, so 20 percent of the pre-human sequence stopped running with a fully green suite [14]. The unit tests are behaving correctly here: they supply the call themselves, which is what makes them unit tests [11].
Scope the payoff before you copy the pattern. Four rounds of adversarial verification produced 19 blocking defects, and three of those were under tests that never went red, so roughly 16 percent of the haul belonged to this category [3][13]. The author separates three claims a suite can make: behavior, reachability, and whether a failure gets reported as the thing it actually was [4]. A wiring test buys reachability, and reachability is all it buys.
For the test to transfer, three things have to hold in your repo. Handlers must be findable by a convention that does not depend on the dispatch site, which here is `check_*_gate` functions inside `*_gate` modules [8]. There must be one dispatch structure to diff against rather than several call sites each registering a subset. And the modules must be importable in a test process without side effects. The failure case worth naming is decorator registration: if a gate registers itself into a global table at import time, the test's own walk populates the table it is about to audit, and the two sets agree regardless of what production imports. The independence the docstring asks for [10] is independence from the production call path itself, delivered by a discovery mechanism that stays genuinely separate from it, not just a second line of code sitting next to the same table.
The reachability incident actually happened. `check_revenue_axis_gate` was implemented and unit-tested, nobody added it to `_MACHINE_GATE_CHECKS`, and a draft declaring affiliate revenue with a subscription call to action in the body went through to AWAIT_HUMAN [5]. That is the gate's exact purpose, and a human reviewer saw an ordinary verdict.
The second category has a worked example in the same post and no test described for it yet. A design decision moved a draft's derived values to a declaration-based representation, where the operator declares the expression and its inputs instead of the gate searching for any arithmetic combination that lands on the number, and that representation had no slot in the ledger's serialization format [12]. The published excerpt breaks off mid-sentence at that point [15]. The shape is familiar: a value present at one end of a serialize and deserialize round trip and absent at the other will satisfy every unit test written on either end, because each end is only ever asked what it does when called [4].
Ranked by verification strength, evidence, and original report placement.
The supplied text of the post breaks off mid-sentence while describing what happened to the declaration-based value in the ledger format.
In the described system a draft is registered, then judged, and the verdict comes back as one of PASS, BLOCK, or AWAIT_HUMAN.
Five machine gates (check_warrant_gate, check_numeral_gate, check_expression_gate, check_disclosure_gate, check_revenue_axis_gate) live in agent_blog/gates/ and run in sequence before a human sees a draft, dispatched from a tuple in verdict.py; every gate had a unit test and every unit test was green.
A verification agent told to stop reading the tests and break the system for real found 19 blocking defects after four rounds, three of which had been sitting directly underneath tests that had never once gone red.
The author argues green unit tests prove behavior (what a function does when you call it) but not reachability, and names three separate claims each needing its own evidence: whether production ever calls it, whether the value survives the trip between call sites, and whether a failure gets reported as the thing it actually was.
check_revenue_axis_gate was implemented and unit-tested but nobody had added it to _MACHINE_GATE_CHECKS, so the real pipeline never called it, and a draft with revenue_axis="affiliate" plus a subscription call-to-action mixed into the body sailed straight through to AWAIT_HUMAN.
Distinct publishers with included, body-backed reporting in this cluster.
dev.to
1 article · August 29, 2026
Follow any of these and your For You feed starts watching them — no settings page required.
build
Deleting guard lines one at a time found 40 of 61 unmeasured by any test1 distinct publisher
build
Force the tool call, then hand Lightsail a long-lived key1 distinct publisher
build
A cache hit is a quota refund: semantic caching with trigrams and no vector database1 distinct publisher
build
Stop timing your GraphQL tests and start counting loader calls1 distinct publisher
Evidence-backed comparisons of source perspectives and observed adoption signals. Read the methodology
Which Builder, Operator, and Investor concerns the observed source mix emphasized—not a truth score.
Evidence, demonstrated adoption, hype gap, incentives, and confidence are assessed independently, each on its own current evidence. How these are measured.
Two pasted runs, one witness
The mechanism is self-checking, which is unusual and worth crediting: the discovery helper and the assertion are printed in full, so a reader can see for herself why walking the gates package catches what tests/test_gates.py structurally cannot, and why an audit that imports _MACHINE_GATE_CHECKS to build its own expectations proves nothing. The figures around that code are a different grade of evidence — 18 passed, 19 blocking defects, four verification rounds — all from a single dev.to post with no repository, commit hash or CI log attached. The argument verifies; the accounting does not.
One repository, and it is the author's
There is nothing to count. The pattern is not a released library with installs, a framework feature with changelog entries, or a practice a second team has described; it is a test file in an unnamed private codebase, reported by the person who wrote it. Scoring uptake from that would be inventing a denominator.
Slightly undersold
The promise is small and the demonstration covers it: one deleted line, one commit, 18 green tests and one failed assertion. If anything the framing gives away ground — the same blindness is shown twice more, in a serialization round trip that silently ate a design decision and an exception handler that made a type error look like policy enforcement, and neither gets the billing the wiring case does. Two things are asserted without hedging and without proof: the defect tally, and a conclusion about verification prompts that the text breaks off before delivering.
Reputation at stake, nothing on sale
No sponsor, vendor, product page or funding sits behind this; the author is republishing his own notes and earns standing if the pattern reads as hard-won. Cutting the other way, the story he chose to tell is that his own gate shipped unwired and that 19 blocking defects survived his own green suite — an unflattering confession, not the shape of promotional writing. The pressure that remains is structural rather than commercial: he is the sole scorekeeper for every number, and grading your own verification is the one part of this a reader cannot audit.
Trust the code, not the count
Confidence splits cleanly along the same seam as the evidence. The structural argument is close to unassailable because it is visible in the source shown — a test that invokes a function cannot notice that production stopped invoking it, and an audit built from the dispatch table it audits passes by construction. The surrounding specifics rest on one unverifiable first-person account of an unnamed system, and the post stops mid-sentence, so we hold the lesson firmly and the numbers loosely.