Build1 publisher3 min readPublished
A coding harness holds the turn open until the repo's own checks exit zero
A dev.to post blames most local-model failure in coding loops on unverified edits and ignored exit codes. It describes a harness that re-reads files from disk and refuses to end a turn while the project's checks are red.
The Engineer · Build desk

What happened
- A dev.to post argues that most of what teams call local-model weakness is the absence of a harness around the model, the scaffolding a frontier model partly supplies on its own.
- The failure it describes: the model announces it edited three files, then half the tests come back red and one of those files is byte-for-byte unchanged.
- After every edit the harness re-reads the file from disk and, when the change is missing, returns "the file is unchanged" to the model as a tool result instead of reporting the task done.
- A non-zero exit from tsc, eslint, ruff, mypy or pytest holds the turn open, so the agent cannot declare work complete while the project's own checks are failing.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- decision If verification closes most of the gap, the spend is engineer time inside your own agent loop instead of a per-token frontier bill, and a team makes that build-or-rent call once.
- constraint Once approvals and wait states live in code, changing agent behaviour needs a code change and a release, so the quick prompt edit stops being an option.
- contradiction The merge list is the post's proof, but with the model unnamed a reader cannot tell how much of that output came from the harness and how much from whatever model was driving it.
The stronger half of the argument is the part about instructions. Teams patch agent misbehaviour by writing more of them: always wait for the command to finish before continuing, never claim completion prematurely, always ask before running destructive commands [13]. According to the post, that approach fails on small models because following a paragraph of procedural instructions is itself a capability weak models lack [13]. So approvals, background command handling, wait states and sub-agent orchestration move out of the prompt and into the harness, where a model that forgets a rule cannot break it [14][15]. "A weak model doesn't need a bigger prompt. It needs a system that catches its mistakes instead of trusting them," the author wrote [4].
Checking the world costs turns. A failed disk verification adds one observation and one model call; a non-zero exit adds another, so a task that hits both needs at least two extra round trips before anything reaches the user [20]. On a rented API that is tokens. On a local GPU it is wall clock. The post defends paying it on error class: wrong code that is flagged is recoverable, and wrong code reported as correct is what ships bugs [11].
For this to transfer to your repo, the checks have to be quick and their exit codes have to mean something. A twenty-minute suite makes a held-open turn a long wait, and a flaky test makes it a loop that will not close [12]. A linter configured to exit 0 on warnings gates nothing. Disk verification has the easier requirement, since re-reading a file and looking for the edit works the same whether the model is a quantized Llama or a frontier endpoint [9].
The evidence offered for the whole approach is a list of merges. The author says several months of work on Atlarix produced code that landed in Remix, Caddy, Traefik and Valkey [16][17]. The post does not identify the model that wrote those patches, or publish pass rates for the same model run with and without the harness [18][19]. It names four mechanisms and describes two of them, the verified edit and the mechanically enforced rule [5][21]. The author's own parenthesis on the merge list, "more on that, honestly, at the end", suggests he has caveats waiting [22]. Until someone runs one model both ways and publishes the numbers, the claim that scaffolding closes most of the gap between a local model and a frontier one rests on one developer's repos.
Parts of the post are worth copying without waiting for a benchmark. Re-reading a file after an edit is a few milliseconds of I/O against a class of failure the post calls the most common one for weak models in a coding loop, the confident false completion [6]. And the post's framing of the frontier-model advantage here is narrow: frontier models make the same false completions, just often enough less that trusting the report mostly works [7].
What to watch
- Whether the author publishes the remaining two mechanisms and a pass-rate comparison of one model run with and without the harness.
- Whether the merged contributions to Remix, Caddy, Traefik and Valkey are identified by model, which would separate harness quality from model strength.
- Whether mainstream coding agents make disk verification of every edit a default behaviour.