Build1 publisher3 min readPublished
A 3,442-byte skill copy in a gitignored .gemini directory became a second source of truth
Prose in AGENTS.md did not stop it, so the project now enumerates agent skills from disk and runs a script that exits non-zero on any copy outside the canonical root, including an empty leftover directory.
The Engineer · Build desk
What happened
- The write-up documents the verification layer in one working project: four agent skills, sixteen repository gates, a hook that runs the gates at edit time, and a narrow mutation testing pilot.
- A 3,442-byte partial copy of one skill turned up in .gemini/skills/, and two weeks later an empty leftover directory was still sitting next to it.
- Both .claude/ and .gemini/ are listed in .gitignore, so the copy never appeared in a diff or in code review.
- The repository now runs npm run lint:skills, which exits non-zero on a copy outside the canonical root, an orphaned reference link, an unclassified ## section, or an entrypoint that routes nowhere.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- constraint Enumerating skills from disk means any similarly named folder is a competing authority, and the same design removes the config list where an extra entry would have shown up.
- decision The gate-or-test call gets a cheap criterion: write a gate when the violation leaves the suite green, and skip it when a test already fails.
- exposure Any team relying on code review to police agent instructions is relying on a process that cannot read files excluded from the repository.
- cost Upkeep for a gate like lint:error-bodies is the list of spellings it knows, and every new idiom in the codebase adds one to that list.
The registry is a directory listing. `fs.readdirSync(SKILLS_ROOT, { withFileTypes: true })`, filtered to directories and mapped to names, is the entire skill set, and no config file lists them [9]. A folder appears, a skill exists. So a folder named like a skill under `.gemini/skills/` registers as a second skill, and nothing compares it against the first. The copy was partial, so the two disagreed about content from the moment it landed [4]. The author wrote that "if the filesystem is the source of truth, any folder with a familiar name becomes a second source of truth" [10].
The entrypoint handles routing. `AGENTS.md` keeps a plain markdown list mapping tasks to skills, so verifying, testing or proving a fix routes to `.agents/skills/shalomut-verification/SKILL.md` [11]. That list exists even for clients that find skills on their own. "Discovery changes how a file is found, never what it says," the author wrote [12]. Inside each skill, a "How to read this skill" section splits the skill's own sections into "always in force" and "open when this condition holds"; the stated rule for the split is that loading a section is cheap and skipping a rule is not [13].
A gate, in this design, is a script that reads the repository and exits non-zero when it finds a violation [15]. Nothing executes: no fuzzed arguments, no generated mutants [15]. `lint:composition` reads where a repository gets constructed, `lint:error-bodies` reads what a response body contains, and `lint:doc-numbers` checks whether a document still agrees with the configuration it quotes [16]. Four of the sixteen gates are named in the write-up [21].
So the transferable part is the criterion. "The tell that you need a gate: the violation is silent," the author wrote, adding that if a violation fails the suite anyway, the test already covers it and the gate is overhead [17]. The audit that produced `lint:error-bodies` counted 21 route handlers interpolating a raw `error.message` into what they sent back, and on `/api/auth/login` that went to anyone [18]. For that 21 to mean anything in your repository, you would need the same TypeScript handler idiom, no shared response serialiser, and no existing test asserting on error payloads. With one response helper, a grep and a fix cover it.
The naive check went in first: a regex for `error.message`. It let `(error as Error).message` straight through, and the audit itself had missed `error?.message` for the same reason [19]. The post breaks off after saying the rule became two rules [20].
A CI job clones the repository, and `.gemini/skills/` is not in the repository, so `lint:skills` can only see a client-local copy when it runs against a working tree, which is where the edit-time hook runs it [22].
What to watch
- Whether the narrow mutation testing pilot widens, and what it catches that the sixteen read-only gates cannot.
- Whether agent clients start writing their config into tracked paths, which would put this class of drift back inside the diff.
- The second half of the error-body rule, and whether it handles spellings beyond (error as Error).message and error?.message.