Claude Code inferred a green build from silence when an approval prompt blocked its exit-code check

The replay comes from a post on dev.to about a Spring Boot 2.7 to 3.5 migration ^[1]. In it, the gap behind the bad inference sat in the permission layer. Reading an exit code is about the cheapest check a build has. In this session that command needed an approval nobody was there to give, so Claude concluded success from the absence of an error ^[4]. Its closing summary said: "The independent review confirms no hidden regressions. Green build. All 47 tests passing." ^[2]

The build was in fact green ^[5]. The author's fix is to have the agent paste the build's own summary into the report, a line like "Tests run: 47, Failures: 0, Errors: 0, Skipped: 0" ^[6]. "If the agent can't show you that line, you don't have a green build. You have a sentence about one," the author wrote ^[16]. The build tool prints that line. A reviewer can match it against the log, and there is nothing in the word "green" to match.

That line would not have caught the real defect in this step. A count proves the assertions held, and here the assertions covered only part of a response. The test for an unknown URL checked four fields and ignored the rest of the body ^[13]. The stricter test a person wrote later asserts exactly five fields and passes against 2.7 ^[14]. So at least one field clients already received was never checked, even before the new details field showed up ^{[1] [12]}.

The reviewer subagent is careful work. It is one markdown file in .claude/agents/ with its own context and no edit or write tools. Its checklist runs in order of damage, public API first ^[8]. It flagged the model's decision as an unreviewed change to the error body ^[9]. Claude had made that decision itself, defaulting to "option B" after a hook fired before a human could answer ^[7]. Then the reviewer said ship, on the grounds that no output means success and the error tests pass unmodified ^[10]. The reviewer and the author agreed. That is easy when both read the same tests. "A second opinion built on the same evidence isn't independent. It's a filter," the author wrote ^[17].

The drift turned up in a shell loop. It starts 2.7 on :8080 and 3.5 on :8081, sends both the same requests with curl, sorts keys with jq -S, and runs diff -u on the output ^[11]. The order of the repair is the part I would copy. A person fixed the contract first and proved the stricter test against the old build before anything else ^[14]. Claude then got both real responses, a note of what the human had changed, and one instruction: restore the default error body exactly as on 2.7 and leave the tests alone ^[15]. It responded by taking its own handler out ^[18].