Build1 publisher3 min readPublished
A dev.to post hands four models a file whose comment contradicts its code, then asks a clean session to fix the inconsistency. It is a well-built way to expose the failure, and no counts are published yet.
The Engineer · Build desk
Compiled by The EngineerSomething wrong?How this is made
A comment and the line below it enter the context window as the same kind of thing: text the repository asserts. Nothing in the file marks the comment as older than the code, and nothing marks it as unverified. The code has a checker in the compiler and the tests. The comment has no checker at all. That is why the `let a = b + 1` edit survives a green build while the line above it still says `make a equal to b` [3]. According to the post, a second session then opens that file, finds the comment is the only statement of intent while the code is merely an observation of behaviour, and puts the code back [4].
Ranking stated intent above observed behaviour is what I want an agent to do most of the time. The post's own framing is that neither model was bad at coding, and that a stale comment made the reasonable move the wrong one [14]. That is the useful part of the argument: more context normally helps, and contradictory context does not, so adding more of it makes the problem worse [8].
For that result to transfer to a real repository, several things have to hold at edit time. The session cannot see the file's git history. No test asserts the new behaviour. The comment is the only written statement of intent. The reviewer skims. The post concedes most of this, listing git history, tests, a ticket and a human as things real agents have, with a parenthetical asking whether you do in fact have tests [15]. That parenthetical is the load-bearing assumption in the whole design.
The multi-session chain is where the mechanism gets expensive. A stale comment is read as intent, an agent writes a test that matches the old behaviour, docs are generated, and a later agent sees comment, test and docs agree and reverts the code with confidence [7]. Three pieces of evidence, one incorrect ancestor. The post calls that shared-error propagation, and the accounting holds: if the test was derived from the comment, the independent agreement count is one, not three [7].
The scale is small and stated plainly: two files, four models, ten runs each, on an Inspect AI task, with local models through Ollama and the Claude models through the API at no more than a few pennies a run [9]. Multiply it out. Two files by four models by ten runs is 80 runs, and each run is two turns, so 160 model calls [16]. The post names the three outcomes it classifies, fixing the comment, reverting the code, or leaving the contradiction alone [11], and publishes no counts for any of them [12]. It does not tell you which model reverts most often, or how often any of them does.
What the mechanism points at is a scope rule, not a documentation policy. If a comment above the changed line asserts an invariant or an access boundary, that comment is inside the change set, and an instruction forbidding you to touch it is an instruction to ship a file that lies. The cost is review noise on every narrow edit. Set against the authorisation example, where the comment still claims admin-only after the condition widened to admit authenticated users [5], I will pay it.
Ranked by verification strength, evidence, and original report placement.
The dev.to post 'Comments considered harmful' argues that developers treat comments as passive documentation for humans, but that for an AI agent they are nothing of the sort: when an agent opens a file, every comment it reads becomes part of its context and shapes what the model believes about intended behaviour, invariants, security assumptions and what should not be touched.
Worked example in the post: a file contains the comment '// make a equal to b' above 'let a = b;'. Given the narrow instruction to change only the executable code so that a != b, a reasonable model output is 'let a = b + 1;' with the comment unchanged. The comment is now wrong, the compiler does not care, and tests may still pass.
In the post's scenario, a different agent later opens the file with no memory of why it changed and is told to fix any inconsistencies. It sees a comment and code that disagree, has no way to know which is true, picks the comment, and puts the code back.
Authorisation example in the post: '// Only admins can access this page' above 'if (isAdmin) { grantAccess(); }'. Product decides authenticated users should also get in, the agent is told to change only the condition, and the result is 'if (isAdmin || isAuthenticated)' with the admin-only comment untouched, so a false comment now describes an authorisation rule.
The post argues a reviewer skimming the file reads the comment and assumes the boundary is still admin-only, and a later agent asked to make sure the implementation follows the documented access policy reads the same comment and removes authenticated-user access, which is a permissions regression introduced by an agent trying to be helpful.
The experiment: two files, four models, ten runs each. The harness is an Inspect AI task. Local models ran through Ollama. The Claude models ran through the API and cost no more than a few pennies a run.
Follow any of these and your For You feed starts watching them — no settings page required.
Evidence-backed comparisons of source perspectives and observed adoption signals. Read the methodology
Which Builder, Operator, and Investor concerns the observed source mix emphasized—not a truth score.
Evidence, demonstrated adoption, hype gap, incentives, and confidence are assessed independently, each on its own current evidence. How these are measured.
One author, method shown, no table
Three of the four models get named counts, and they are lopsided ones, but they arrive as sentences in a 'How to read it' walkthrough with no table, no run artefacts and no repository link; the fourth model is never identified. The method itself is stated well enough to rebuild, which is why this sits above anecdote and short of a checked result.
Harness only, no field evidence
Nothing here has been observed outside the harness: two toy files and eighty runs, with the author saying plainly that git history, tests, the ticket and the human were taken away on purpose. The revert counts are real measurements of real models, and no team, codebase or tool vendor has reported the same failure in working software.
Framing runs ahead of two files
Calling a stale comment 'a dangerous lie' is heavier than two toy files can carry, and the step from a stripped bench to production agents belongs to the author rather than to his data. The gap stays small because the reported effect is not marginal, ten of ten reverts from Haiku 4.5 and eighteen of twenty from Qwen3-coder, and because the piece raises the three obvious objections against itself before anyone else can.
Personal post, nothing on sale
A community-platform post with no tool, course or service attached; the models tested belong to other people, and the one that occasionally gets it right is a paid Claude tier, which is not how a stacked deck usually reads. The pressure that does exist runs toward the premise: narrow-edit prompting has to be common for the finding to matter, and that comes from the author's experience rather than any sample.
Clean design, unreplicated counts
The two-turn design isolates the mechanism cleanly, and the mechanism is easy to believe: one file, a fresh session, and a comment that disagrees with the code. Confidence stays mid-scale because every number comes from a single unreplicated run by the person making the argument, the Opus 5 authorisation cell never finished, and the author's own concession stands, that with nothing else in context the model's choice is defensible, which makes the result partly a measure of the prompt.
build
Coding agents cost $4,125 a month because 73% of it is context you already sent1 publisher
build
OpenClaw makes the channel the architecture, and the reasoning loop a lodger1 publisher
build
Claude's system prompt grew ninefold in two years. Version yours like code.1 publisher
build
Agent reliability is a harness problem, not a prompt problem1 publisher
Publishers with included, body-backed reporting in this cluster.
1 article · September 8, 2026