Build1 distinct publisher3 min readPublished
A dev.to triage procedure sorts every rejected agent patch into deterministic regression, fixture drift, or flake, and freezes only the flake. The published script is worth reading first, because its fixture check is a placeholder.
The Engineer · Build desk

Compiled by The EngineerSomething wrong?How this is made
Read the artifact before you adopt the procedure. In the published `triage_gate.py`, `fixture_hash()` returns the first twelve characters of the sha256 of the test id, with a comment saying a real setup would hash the fixture files for that test [15]. Follow that into `classify()`. The recorded hash defaults to the current one when the test id is absent from the ledger [16], and every non-flake verdict writes the hash back under the same key [17]. Substitute the stub into the comparison and `current != recorded` is false on both paths: a new test is compared against itself, and a known test is compared against a value computed from the same string [3]. Class B is unreachable as shipped, so drift lands in the bucket the script labels deterministic regression [3]. That function is the line to write before you switch the gate on.
`RETRIES` is a module constant set to 3, and `QUARANTINE_DAYS` to 7 [14]. Three is generous for the cheap case and thin for the interesting one. A test that fails half the time comes up red three times running with probability 0.5 cubed, or 12.5 percent [2]. One coin-flip flake in eight therefore clears the re-run stage, clears the fixture check, and gets written down as a regression in the agent's code. Five re-runs would take that to 3.1 percent [4] and cost two more executions. Which of those you want depends on how long the test takes and how much you trust a verdict that sends work back to a model.
The quarantine entry carries a first-seen timestamp, a reason, and an expiry [10], seven days out by default [14]. Nothing in the script reads the expiry back, and the skipping of quarantined tests belongs to a pipeline the script does not include [18]. A timestamp no job ever reads is a to-do list with better formatting.
The other adoption cost is enumeration. Step two hashes the fixtures a test loads and compares them against the value at the last green commit [8], which assumes you can name those fixtures per test. Where snapshots and seeds are generated at session scope, that mapping is the project and the script is the easy part.
I would still take this trade in my context, which is a hosted pipeline where agent patches arrive faster than anyone reads logs. The value is not the code; it is that an unlabeled red build becomes one of three named jobs, which is impossible from the first log line alone [6]. I would build it in roughly the reverse of the article's order: real fixture hashing first, then the ledger and its reader, then tuning the retry count.
Ranked by verification strength, evidence, and original report placement.
The procedure classifies every rejected patch as Class A (deterministic regression: fails on the first run and every re-run with the same input, the only class where the agent's code is the primary suspect), Class B (fixture drift: a fixture ID, snapshot or generated seed changed outside the patch), or Class C (flake: re-runs flip between red and green).
In the procedure, Class A is the only class where the agent's code is the primary suspect, and under Class B the agent patch may be innocent, with the prescribed action being to regenerate the fixture and leave the patch untouched.
For Class C flakes the suspects named are timing, ordering, or shared state, not the patch.
The article states that all three failure classes look identical in the first CI log line, and that the only way to separate them is a controlled re-run.
Step one of the procedure re-runs the exact failing test three times with the same seed and the same command: all green makes it a Class C candidate, mixed results make it Class C, all red continues to the next step.
Step two hashes the fixtures the test loads and compares the hash with the value recorded at the last green commit; a mismatch means Class B, and the response is to regenerate the fixture and re-run.
Distinct publishers with included, body-backed reporting in this cluster.
dev.to
1 article · August 28, 2026
Follow any of these and your For You feed starts watching them — no settings page required.
build
An AI test suite hit 94% coverage and missed the one branch that mattered1 distinct publisher
build
Your 90% Cache Hit Ratio Is a Lagging Indicator. Alert on Cold Misses Per Key1 distinct publisher
build
SSE promises framing, not JSON: the streaming bug that only appears on long answers1 distinct publisher
build
Sequence-level equivalence catches the cache a single-call test suite waves through1 distinct publisher
Evidence-backed comparisons of source perspectives and observed adoption signals. Read the methodology
Which Builder, Operator, and Investor concerns the observed source mix emphasized—not a truth score.
Evidence, demonstrated adoption, hype gap, incentives, and confidence are assessed independently, each on its own current evidence. How these are measured.
Checkable on the page, untested in the field
Almost everything here can be verified by reading rather than by trusting the author: the taxonomy, the step order, the shape of the quarantine file and the entire script are printed, which is why our two sharpest findings come from the code itself. What the reporting cannot offer is anything from outside one engineer's practice - no distribution of how often failures really land in each bucket, no before-and-after on triage effort, and 'every failure I have triaged' doing the work that data would.
No usage signal
Not one user, repository, run count or team appears anywhere in this reporting. triage_gate.py is presented as a minimal reference implementation and dev.to's post is the only place it exists, so there is nothing to measure uptake against and we decline to invent it.
Prose runs ahead of the script
The post calls triage_gate.py the reproducible artifact of the procedure and states flatly that CI skips quarantined tests. As published, the fixture comparison hashes the test's own name - so Class B, one of the three pillars the scheme stands on, can never fire - and no code reads the expiry the script writes. The gap stays moderate rather than wide because dev.to volunteers its own main caveat: three runs is a sample, not a proof.
Disclosed sponsor placement
The closing section is the tell, and to its credit it says so out loud. The loop's 'two boring jobs' get re-described as tiny, mostly idle workloads that happen to want hosted compute and a model to draft failure summaries - precisely the shape of MonkeyCode's free server option and free model access, after which dev.to discloses that the whole post was prepared as MonkeyCode's product outreach. The technical content stands on its own; the workflow was drawn around a sponsor.
One self-published account, verifiable code
Self-published, unreviewed and unaccompanied - a single dev.to post with no corroboration anywhere. What keeps our reading steady is that the artifact either does what the prose says or it does not, and in two places it does not. Confidence in the taxonomy as advice is a different matter, and rests on one practitioner's word.