Build1 distinct publisher3 min readUpdated
A dependency rule for agent config files read only the scalar form of a YAML key. Authors who declared nothing went clean; authors who declared properly got the warning.
The Engineer · Build desk
Compiled by The EngineerSomething wrong?How this is made
A linter that fails CI when agent config files bake in machine-specific assumptions had a rule requiring authors to declare any external CLI they invoke, and its implementation read only the scalar form of the frontmatter key that declaration lives in [1][2][4]. According to the maintainer's write-up on dev.to, the consequence was a clean inversion: authors who never wrote a `requires:` key at all were never flagged, while authors who sat down and wrote the contract in the YAML block list form were told they had not declared their dependency [5][6].
The mechanics are dull, which is the point. `requires: codex` parses. The block list, which is what you write the moment you have two of anything, did not [2][3][4]. A validator that accepts fewer shapes than its format legitimately permits does not fail gently and evenly. It fails against exactly the set of users who engaged with the rule, because non-compliance and unparsed compliance are the same byte sequence to the checker: absence. The maintainer says this shipped in a patch release and was found only after a commenter used the phrase "dependency contract," prompting a re-read of the implementation [7].
The same shape then recurred twice in a different rule. `unverified-write` flags a file that changes external state, such as `git push`, `npm publish` or an `INSERT`, and never reads that state back [8]. It was measured against 586 real skill files from a public registry, with two false-positive shapes found and fixed, ending at a 0.7% fire rate with every hand-checked hit genuine [9] - roughly four files out of 586 [21]. A different model, asked for a pre-publish read, produced a failing input in about a minute: "Never run `git push --force` from this skill." [10] That is a push in a code span in a file with no read-back, so the rule fired [10]. Writing down "don't push without asking" is, per the maintainer, the most common act of care in that genre of file [11]. After prohibitions were excluded, permission sentences such as "Only run `git push` when the user asks" still fired [12].
Three releases, three variants, all leaning the same way [13]. The stated reason is that a text-matching rule sees mentions, not actions, and mentions of a dangerous operation are not evenly distributed [14]. The careless file does not contain the string at all; the careful file contains it three times, twice in ways that are not the thing being detected [15]. So every false positive is drawn from the conscientious pool, which is also the pool most likely to read the warning and uninstall over it [16]. Static analysis already names this as use versus mention, and the older CLI rule ignores a bare `codex` in prose while firing on `codex exec build` - an exclusion written two releases before the new rule was built without it [17][18].
The 586-file corpus could not have caught any of it [19]. Published, downloadable skills are written to be used and say "run this" far more than "never run this"; the prohibition shape lives in team-internal AGENTS.md files nobody uploads [19]. The data was real and biased in precisely the direction that hid the failure [20].
Two things worth checking in your own validators. First, enumerate the shapes your input format permits for every key you read, and test the ones your fixtures do not use. Second, ask whether the corpus you validated against is drawn from the population you police, or only from the part of it that publishes.
Follow any of these and your For You feed starts watching them — no settings page required.
Ranked by verification strength, evidence, and original report placement.
The author maintains a linter that reads agent config files (SKILL.md, AGENTS.md, CLAUDE.md) and fails CI when they bake in something that only works on the author's machine.
One rule requires that if a file calls an external CLI, the author declares it in frontmatter, e.g. 'requires: codex'.
Anyone with more than one dependency writes the YAML block list form (requires: followed by '- codex', '- gemini').
The implementation only read the first (scalar) shape, so the block list was invisible to the linter, and it warned authors for an undeclared CLI they had in fact declared.
Authors who ignored the dependency question entirely were never flagged, because they never wrote a 'requires:' key at all.
Authors who wrote the contract properly received a warning telling them they had not; the rule was inverted with respect to the behaviour it was trying to encourage.
Evidence-backed comparisons of source perspectives and observed adoption signals. Read the methodology
Which Builder, Operator, and Investor concerns the observed source mix emphasized—not a truth score.
Evidence, demonstrated adoption, hype gap, incentives, and confidence are assessed independently, each on its own current evidence. How these are measured.
Specific but single-source and unverifiable
The account is unusually concrete for a personal postmortem: named file shapes, a quoted false-positive input, an explicit corpus size (586 files), a measured fire rate (0.7%) held constant across the fix, and a self-identified regression against an exclusion the author had written two releases earlier. Everything, however, rests on one first-person article from one publisher; the linter, its repository and the registry are unnamed, no dataset or reproduction path is published, and the population-level assertions about prohibition-sentence frequency and conscientious-author skew carry no measurement.
No usage evidence supplied
The supplied source documents releases of the linter's rules and a benchmark run over a registry corpus, but discloses no installs, downloads, dependent repositories, users or third-party deployments, and does not name the tool. The 586 skill files measure the size of the corpus the rule was tested against, not adoption of the linter. Adoption therefore cannot be scored without guessing.
Broadly aligned, mild over-generalization
The framing is confessional rather than promotional: the author reports shipping an inverted rule, concedes the corpus was 'the wrong real data', and re-measures to show the exclusions did not delete the signal, which if anything under-sells the work. The small positive gap comes from generalizing beyond the evidence — a base-rate claim about conscientious authors and uninstall behaviour, and a universal prescription for any detector running on unseen files, both extrapolated from one unnamed project's three releases with no measured population data.
Maintainer self-report, reputational upside
The author is the maintainer of the tool being discussed and is publishing on a developer platform where postmortems attract engagement; the piece also grew out of comments on his own prior post, and the new rules are his own product surface, so there is reputational and distribution incentive. Offsetting this, the article discloses defects in the author's own shipped releases rather than claiming wins, names no commercial product, price or sponsor, and no vendor or funding relationship appears anywhere in the supplied material.
Moderate on mechanics, low on generality
Confidence is moderate that the described defects and fixes happened as narrated: the technical story is coherent, mechanism and remedy match, and the reported numbers are consistent with each other. It is low that the wider conclusions hold generally, because the cluster has a single publisher and a single self-reported source, the project is unidentified, adoption is unmeasurable from the material, and the strongest interpretive claims are unquantified.
build
A "Done." is a claim about the world, not a sentence you can grade1 distinct publisher
build
The Context Tax: Your Developers Are Doing Unpaid Platform Work Every Session1 distinct publisher
build
Before you spend quota on an agent skill, make it pass an eval harness1 distinct publisher
build
The AGENTS.md file is an audit of the documentation you never wrote for humans1 distinct publisher
Distinct publishers with included, body-backed reporting in this cluster.
dev.to
1 article · August 17, 2026