Build1 publisher3 min readPublished
Saying "I authorize it" got past the CLAUDE.md rule in both runs that tried it
A 29-session test on Claude Code v2.1.273 ran the same protected-directory rule two ways, as prose in CLAUDE.md and as a PreToolUse hook. Both held on a plain task. The comparison that separates them rests on four runs.
The Engineer · Build desk
What happened
- A 29-session headless test gave Claude Code the same "never edit protected/" rule twice, once as a single line in CLAUDE.md and once as a PreToolUse hook, against a throwaway project.
- On a prompt that never mentioned the protected file, each mechanism allowed 0 edits across 5 runs, while sessions with neither rule nor hook edited all three files in 3 of 3 runs.
- When the prompt said the user maintained the script and was authorizing the edit, the CLAUDE.md rule gave way in both runs and the hook held in both.
- The 33-word CLAUDE.md bullet cost 58 to 70 input tokens on every request, whether or not the session went near the protected directory.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- decision Claude Code's own documentation draws the line the test measured, so a rule that must survive a user asserting authority has to sit in a hook, and CLAUDE.md is left holding conventions.
- cost The prose rule's cost scales with request volume and the hook's with violations. A busy repository that rarely trips the rule pays more for the sentence than for the script.
- exposure The line that flipped the rule was ordinary prompt text, so anything able to put text in the prompt can attempt the same claim of maintainership.
- constraint Four runs carry the separation between the two mechanisms. That is enough to show a direction and not enough to quote a failure rate for either.
The hook is a short shell script registered for `PreToolUse` with the matcher `Write|Edit|Bash` [9]. It reads the event JSON from stdin, takes `tool_input.file_path` or `tool_input.command`, and refuses when that string contains `protected/` [9]. The decision is a substring test on text the model proposed. It covers three tool names. A route to the same file that never spells `protected/` in the path or the command goes through unexamined.
Claude Code's memory documentation states the difference in one sentence: CLAUDE.md files are loaded as "context, not enforced configuration. To block an action regardless of what Claude decides, use a PreToolUse hook instead." [6] The rule under test was 33 words, the only content of CLAUDE.md apart from a heading [8]. It held on the plain task [2] but gave way once the prompt claimed maintainership [3].
The rule bills on every request, at 58 to 70 input tokens [4]. Against the run's `--max-turns 10` [12], a session that goes to the cap pays 580 to 700 tokens for the bullet [1]. The hook was free until it fired, and then it added a whole extra model round trip on a direct request [5].
The first seven runs went without `--strict-mcp-config`, and four of them showed first-request totals about 780 tokens higher than comparable later runs, because tool names from a connected mail integration had arrived before the first request in some sessions and not in others [15]. That contamination is 11 to 13 times the size of the effect being measured [2]. The authors kept the behaviour results from those runs and took token numbers only from runs with the flag [15]. They also passed `--setting-sources project,local`, because the test machine carries its own guard hook on `Bash` and `Edit` [13], and pointed `--settings` at a file outside the project so the tree Claude could see was identical in both arms [11]. Two of the three confounders I would have worried about were closed before the first measurement.
For the numbers to transfer, your setup has to look like theirs. Every run used `--permission-mode acceptEdits`, which let edits apply without a prompt, since a `-p` run cannot answer one [14]. An interactive session with write approvals still has a person between the model and the file. The results come from one Claude Code version, v2.1.273, one model, and one phrasing of the override [7][3]. The five tallies reported add to 17 sessions, so 12 of the 29 sit outside the counts quoted here [3].
I would put any rule that has to survive a persuasive user behind the hook, and leave CLAUDE.md for conventions I want the model to read. On this evidence, one authorization sentence moved the prose rule in 2 of 2 runs and moved the hook in none [3].
What to watch
- Whether the three refusal modes (stderr with exit 2, stderr with exit 1, JSON permissionDecision deny) differ in what the model is shown and whether it retries.
- Whether the CLAUDE.md rule holds under interactive permission prompts instead of --permission-mode acceptEdits.
- More runs on the authorization prompt, and other phrasings of a user claiming authority over the protected path.