CPython's mtime-plus-size cache key hides a byte-identical mutation from the test suite

The import path is short enough to audit by hand. Python reads the header of the cached `.pyc` before it compiles anything, and that header holds the source mtime in whole seconds plus the source size ^[1]. If both fields match, the cached bytecode loads and no compile happens ^[2]. Flipping `<` to `>` changes neither field ^[1].

Three conditions have to hold together for that to reach a mutation run: a `.pyc` sitting beside the source, an edit that preserves byte size, and an mtime seconds field matching the header's. The author reproduced it by compiling a pyc from `a == b`, overwriting the source with `a != b` and forcing the mtime back; the original comparison ran on the first try ^[5]. Forcing the timestamp by hand is not the only route there. A harness that writes its mutant inside the same second the pyc was compiled meets the timestamp condition for free ^[3].

The old canary was blind to all three conditions by construction. It overwrote the module with unparseable garbage, and garbage has a different byte length, so the pyc really did invalidate, the suite really did fail, and the check reported success ^[4].

Direction matters if you read kill rates off a tool like this. A mutant that runs original bytecode passes the suite and is filed as surviving ^[3]. A false survival counts as a mutant the tests missed, so the measured kill rate lands below what the tests earn ^[2]. The nine bugs in the author's earlier post all leaned the other way, in favour of his results ^[18].

None of it reaches the harness in question. Every temp-copy call site already passes `ignore_patterns("__pycache__", "*.pyc", ...)` ^[6], and a target with 19 real pyc files copied across with zero of them arriving ^[7]. That argument was written to avoid copying junk. "The reason an instrument is correct is often not the reason you wrote the code that makes it correct," the author wrote ^[8]. The fix is a second canary that preserves byte size, running alongside the original: eight of twelve targets are eligible and all pass, and the remaining four report N/A instead of being skipped quietly ^[9].

The second reader finding came from Ahmet Ozel ^[19], who suggested keeping a deliberate negative control, a case where the score should be near zero and a high score is the alarm ^[10]. His reading of the earlier post was that "debugging is triggered by surprise" explains a one-directional bias without anyone needing to be dishonest ^[13]. The author built the control out of vacuous tests that import the module, call things and assert nothing meaningful, and it killed 7 of 51 mutants, 13.7 percent ^[11]. All seven kills came from the operator that makes a function return None; the suite caught zero constant mutations and zero comparison mutations ^[12].

So the single-character comparison flip is both the mutation that preserves byte size ^[1] and the mutation a suite asserting nothing never catches ^[12]. A control also has to fail loudly when it passes for a boring reason, because 0.000 is what you score when the module never loaded, and one of the original nine bugs produced three targets at 0.000 that read as a finding ^[16]. The control now asserts separately that the suite passes on clean source ^[17].

Ozel's other point lands on a practice the author was already following. Writing down an expected number, he argued, "does nothing when the instrument is broken in a way that produces the number you predicted" ^[14]. One of the nine was exactly that: a classifier read `self.assertEqual` as "no assertion" while the hypothesis under test was that models write assertion-free tests ^[15].