Skip to content

Build1 publisher3 min readPublished

The linter that passed everyone who ignored it and warned everyone who complied

A dependency rule for agent config files read only the scalar form of a YAML key. Authors who declared nothing went clean; authors who declared properly got the warning.

The Engineer · Build desk

Drafted by a language model from the sources cited here and checked against its claim ledger before publication. How we use AISend a correction

What happened

  • The author maintains a linter that reads agent config files (SKILL.md, AGENTS.md, CLAUDE.md) and fails CI when they bake in something that only works on the author's machine.
  • One rule requires that if a file calls an external CLI, the author declares it in frontmatter, e.g. 'requires: codex'.
  • Anyone with more than one dependency writes the YAML block list form (requires: followed by '- codex', '- gemini').
  • The implementation only read the first (scalar) shape, so the block list was invisible to the linter, and it warned authors for an undeclared CLI they had in fact declared.
  • Authors who ignored the dependency question entirely were never flagged, because they never wrote a 'requires:' key at all.

Compiled by The EngineerSomething wrong?How this is made

Why it matters

A linter that fails CI when agent config files bake in machine-specific assumptions had a rule requiring authors to declare any external CLI they invoke, and its implementation read only the scalar form of the frontmatter key that declaration lives in [1][2][4]. According to the maintainer's write-up on dev.to, the consequence was a clean inversion: authors who never wrote a `requires:` key at all were never flagged, while authors who sat down and wrote the contract in the YAML block list form were told they had not declared their dependency [5][6].

The mechanics are dull, which is the point. `requires: codex` parses. The block list, which is what you write the moment you have two of anything, did not [2][3][4]. A validator that accepts fewer shapes than its format legitimately permits does not fail gently and evenly. It fails against exactly the set of users who engaged with the rule, because non-compliance and unparsed compliance are the same byte sequence to the checker: absence. The maintainer says this shipped in a patch release and was found only after a commenter used the phrase "dependency contract," prompting a re-read of the implementation [7].

The same shape then recurred twice in a different rule. `unverified-write` flags a file that changes external state, such as `git push`, `npm publish` or an `INSERT`, and never reads that state back [8]. It was measured against 586 real skill files from a public registry, with two false-positive shapes found and fixed, ending at a 0.7% fire rate with every hand-checked hit genuine [9] - roughly four files out of 586 [21]. A different model, asked for a pre-publish read, produced a failing input in about a minute: "Never run `git push --force` from this skill." [10] That is a push in a code span in a file with no read-back, so the rule fired [10]. Writing down "don't push without asking" is, per the maintainer, the most common act of care in that genre of file [11]. After prohibitions were excluded, permission sentences such as "Only run `git push` when the user asks" still fired [12].

Three releases, three variants, all leaning the same way [13]. The stated reason is that a text-matching rule sees mentions, not actions, and mentions of a dangerous operation are not evenly distributed [14]. The careless file does not contain the string at all; the careful file contains it three times, twice in ways that are not the thing being detected [15]. So every false positive is drawn from the conscientious pool, which is also the pool most likely to read the warning and uninstall over it [16]. Static analysis already names this as use versus mention, and the older CLI rule ignores a bare `codex` in prose while firing on `codex exec build` - an exclusion written two releases before the new rule was built without it [17][18].

The 586-file corpus could not have caught any of it [19]. Published, downloadable skills are written to be used and say "run this" far more than "never run this"; the prohibition shape lives in team-internal AGENTS.md files nobody uploads [19]. The data was real and biased in precisely the direction that hid the failure [20].

Two things worth checking in your own validators. First, enumerate the shapes your input format permits for every key you read, and test the ones your fixtures do not use. Second, ask whether the corpus you validated against is drawn from the population you police, or only from the part of it that publishes.

Loading claim ledger
Loading source directory links
Loading share composer
Loading topic controls
Loading related stories