Build1 publisher3 min readPublished
The mutation gate skips every pattern it cannot find in the target file
A dev.to workflow makes every recorded golden prove it can fail before a refactor starts. The gate script that enforces it drops any of its three hardcoded patterns the target file does not contain.
The Engineer · Build desk

What happened
- A dev.to post sets out a legacy workflow in which you record golden values, then prove each recorded case can fail under a deliberate mutation, and only then make the smallest safe change.
- Its capture harness replaces charge_card and send_receipt with spy closures that log the call name, repr'd arguments and sorted kwargs to a module-level ledger and return 0.
- Replay is a single parametrized pytest asserting exact equality between a fresh snapshot and the committed goldens.json, and the comparison does no fuzzy matching.
- The gate script rewrites app/legacy.py inside a temporary copy of the repo, runs the golden tests there, and exits 1 when any mutant leaves the suite passing.
- The post tells readers to treat the survivor counts its script prints as illustrative, because the real number depends on the function under test.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- constraint Anyone adopting the script as written inherits a gate whose strictness tracks their coding idioms more than their coverage, since a file that never spells `return total` or `total = 0` passes untouched.
- contradiction The rule is stated per recorded case while the script judges the suite as a whole and stops at the first failure, so the untested case the gate exists to find stays invisible to it.
- cost The cost of adopting it scales with the number of mutants: every pattern that matches means a full working-tree copy and another run of the golden file.
- capability Because the golden holds the ordered call ledger, an extraction that preserves the return value but reorders the writes now breaks the replay.
The seam advice comes in the right order. Time, randomness, network calls and global writers make goldens flaky, and the post says to patch those seams in the harness, in test code, without refactoring first [16]. The recorded values then go to disk with sort_keys and get committed, because a diff a reviewer can read is better than a magic assertion [8].
The spies return 0 on every call [5]. If settle_order branches on what charge_card returned, the golden captures the stubbed path. The post handles that by hand: run the capture once against the untouched file, read the JSON, and delete any case whose behavior looks like an artifact of the harness [17]. snapshot() assigns those spies onto the legacy module, and its finally block contains only pass; the post does not show the originals being restored [6]. After the first case runs, the module stays patched for the rest of the process.
The gate itself is run_gate: it reads app/legacy.py, calls re.subn once per pattern, and skips to the next pattern when hits is 0 [12]. The list is three literal rewrites: `if qty <= 0:`, `total = 0` and `return total` [11]. survivors only grows when the pytest run inside the mutated copy returns 0 [14]. Point the script at a legacy file that uses none of those three spellings and survivors stays empty, the script exits 0, and pytest is never invoked [2]. A run that kills all three mutants exits 0 too.
The premise holds up: "A characterization test that cannot fail is decoration, not a safety net," the post says [2], and its rule is stated per case: "Every characterization case must survive one deliberate, minimal mutation of the code under test" [3]. The script measures something else. It runs tests/test_goldens.py with -q -x [13], so the first failing case ends the run, and a kill tells you some case caught the mutant without telling you which one [3]. The script's output leaves out any recorded case that never fails under any of the three patterns [4].
Cost follows the mutant list: each pattern that matches copies the working tree into a temp dir, minus .git, .venv and __pycache__, then runs the golden test file there [13], so three matching patterns mean three tree copies and three suite runs [1]. With four recorded cases that loop is short [4]. It grows as your suite runtime multiplied by the number of mutants you write.
For the counts to mean anything in your repo, the mutants have to be written against your own control flow, and the post says as much: treat the printed numbers as illustrative, because the real number depends on your function [18]. It also says the expensive part is enumerating branches, not writing asserts [18]. One detail of the implementation: re.subn substitutes over the file's text, so `return total` matches inside a comment as readily as inside a function [5].
What to watch
- A check that fails on a zero-hit pattern would close the path to a green exit with no pytest run at all.
- Pairing each killed mutant with the case id that caught it, which -q -x cannot do, would make the per-case rule enforceable.
- The published post breaks off where it hands branch enumeration to a model, and stops before saying what it asks the model to produce.