Deleting a 1,350-line CLAUDE.md broke two rules out of twenty

A developer removed a 1,350-line CLAUDE.md from a working repository, ran the same ticket-sized task thirty times, and found that two rules out of twenty depended on the file existing ^[1][5]. That matters because the standard advice about agent rules files - that they are advisory rather than enforcement, that long ones get ignored, that anything buried deep gets skipped and belongs in a hook - is repeated everywhere and, according to the dev.to writeup by Mayank Kaul, has never been measured ^[3][4]. The design: twenty checkable rules, one task, six configurations (full file preloaded, file removed entirely, file on disk with auto-loading suppressed, nineteen lines plus a Skill, and two more arms on a stripped-down brief), with a deterministic checker reading the resulting file tree ^[5]. With the file present, 399 of 400 rule checks passed ^[6]. That works out to twenty runs with the file and ten without, and a single failed check across every arm that had it ^[1][2]. Seventeen of the twenty rules scored five out of five in every configuration, including the arms with no file at all ^[7]. Across six hundred checks only three rules failed anywhere, and two of them account for the entire gap ^[8]. Those two are not about length. Rule 12 says put a bullet under ## Unreleased; the repository contradicts it twice, since that heading has held the placeholder "- Nothing yet." for the project's whole history while all eight released sections carry the actual bullets, and the changelog's own header instructs you to cut a new version section and move the patch digit on every merged change ^[9]. Every run without the file read that header and followed it ^[10]. One reported moving to 0.4.3 "per the changelog's stated rule that the patch digit moves on every merged change" ^[11]. Rule 13 failed the same way: the file says bump the patch digit, the history shows 0.4.0 was a minor bump because it added new API surface, and three runs that added a public module picked 0.5.0, while a fourth left the version alone ^[12]. Kaul notes these are the only two rules in the set where the file contradicts what the repository demonstrates ^[13]. The burial claim does not survive the layout. Rules eleven through twenty all sit below line 314, under a heading in which the file itself says these are the ones most often skipped in practice ^[14]. Seven of those ten never failed anywhere, and rules 12 and 13 failed only in the arms without the file, which cannot be a burial effect because burial requires the file to exist ^[15]. The one remaining buried-rule failure did occur in a run that had the file, though the reporting is cut off before the detail ^[23]. The instrumentation notes are the part worth stealing. The first version of the experiment found no effect at all, because deleting the file left it in git and git status printed D CLAUDE.md in the first orienting command; ten of ten runs recovered it and six said outright they had read the contract out of HEAD ^[16]. Two further channels leaked: a memory plugin installed months earlier and forgotten, and Claude Code's own per-project memory directory ^[17]. The base repository was rebuilt so the file was never committed at any depth, verified by scanning every blob in the object store ^[18]. Kaul names the residual confound himself: the test harness carries a comment pointing at CLAUDE.md, four of the ten no-file runs reported the file missing, and one said it followed the conventions the committed modules demonstrate instead - so those arms are a model told a contract exists and unable to reach it, not a naive one ^[19][20][22].