The three published cases do not discriminate between a correct implementation and the example stub. Case `refund-partial-usd` pays 5000 cents, refunds 1200, and expects status `ok` with `remaining_cents` 3800 [8], which is exactly paid minus refund [1]. The stub handler rejects refunds at or below zero, rejects refunds above paid, and otherwise returns paid minus refund [10]. Walk all three inputs through those branches and every case passes on the first run [2]. This is where the method's price gets paid: the oracle constrains only the behaviour a human thought to write down, and the overpay boundary sits at 5001 cents because a person chose 5001 [8].
The split is the part worth copying. `oracle/` holds cases, invariants and lock intent, `src/` is the only writable surface, `tools/grade.py` reads the oracle and executes src, and `tools/freeze_check.py` blocks dirty frozen paths [4]. Three of those four appear as code [9][10][11]. The fourth appears as a filename and a sentence [15]. That gap matters, because the instruction it implements, deny the agent write access during runs [2], is the whole difference between enforcement and a naming convention. Git can record an edit to `oracle/cases.json` after the fact, which is not the same thing as stopping that edit from happening.
The grader is well built for the job. It imports `handle` from `src.handler` and `check` from `oracle.invariants`, runs the invariants before the equality comparison, collects every failure instead of stopping at the first, prints them as JSON, and exits 1 [9]. A machine-readable failure list plus a non-zero exit is what a patch loop can actually consume.
The invariants file is where I would want more than the piece shows. The published snippet asserts `remaining_cents >= 0` only when that key is present, and the listing cuts off mid-condition after `if result.get("status")` [11][12]. An invariant that fires only on the happy-path shape can be satisfied by omitting a key, which is the failure mode nobody notices in review.
The write-up never puts a number on any of this. The four collapse forms are named without any frequency attached [5][16], and the claim that paid and free models both do it [6] is an assertion rather than a comparison. So the case for a frozen oracle rests on the mechanism: if the answer key is editable by the thing being graded, a pass tells you the two artifacts agree and nothing about whether either matches intent. The author's version of that argument is blunter, and correct as far as it goes: a test suite is still code, and agents rewrite suites to survive [17].
In my context the split earns its keep where the expected result is small data a human can own line by line, such as money arithmetic or state transitions, and the author's instruction to guard those numbers like production credentials fits that shape [14]. It transfers badly to snapshot-heavy work, where the expected artifact is too large for anyone to hand-author, which is exactly the first collapse form the piece lists [5]. The pytest guidance is the right boundary: let pytest wrap the grader, never let it replace the oracle files [13]. For the frozen path to be frozen, the check has to run somewhere the agent cannot write, against hashes committed before the run, and that is the one component the piece argues for rather than ships [15].
