Build1 publisher3 min readPublished
The bug that was not in the diff: 1,100 double charges cleared a four-minute review
A retry loop with backoff, a timeout and test coverage charged 1,100 customers twice because it read an HTTP timeout as proof nothing happened. The diff never said otherwise.
The Engineer · Build desk
What happened
- Two engineers approved a payment retry loop in under four minutes; it carried backoff, a max attempt count, a context timeout and test coverage.
- By 6 AM the same day, 1,100 customers had been charged twice.
- DORA's 2024 data tied every 25% rise in AI adoption to a 7.2% fall in delivery stability, while function-level quality metrics rose about 3.4%.
- GitClear's 2026 study of more than 600 million commits found duplicated code blocks up 81% since 2023.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- constraint Output scaled about tenfold and comprehension did not, so the backlog gets cleared by loosening what counts as reviewed rather than by reading faster.
- exposure With error-masking constructs up 47%, the failures that once surfaced loudly now return quietly, and a quiet failure is exactly what a retry loop treats as permission to charge again.
- decision An agent reviewer covers the mechanical class and cannot cover the contract class, which leaves the buying question as who is named for knowing the service invariants, not which bot to license.
- contradiction Per-function quality dashboards improve through the same period stability degrades, so the number that clears a merge is not the number that pages an on-call engineer.
An idempotency key is not a property of a diff. It lives in the contract between caller and receiver: the caller sends the same key on every attempt, the receiver refuses to act on it twice. Nothing inside a retry loop's own file records whether that agreement exists. So the review, human or model, checks what the file can express, and here every one of those checks passed: backoff present, attempts bounded, context timeout set, tests green [2]. The code was wrong anyway, because it read an HTTP timeout as evidence that nothing had happened rather than evidence that the caller does not know what happened [5]. Catching that required knowing something the diff did not contain [6].
Two engineers signed off in under four minutes [3]. The loop then ran from 2:47 AM until the count was taken at 6 AM [1][4], about three hours and thirteen minutes [2]. Divide the outcome by the inspection and you get roughly 275 duplicate charges for every minute a human spent reading [1]. That ratio is the economics: writing got about ten times cheaper, reading got no cheaper at all, because comprehension is capped by how long one person can hold attention in a day, and the queue is then managed by dropping the standard until it fits [7].
GitClear's 2026 pass over more than 600 million commits gives that drop a fingerprint. Moved code, which is what refactoring looks like in a commit, fell from 21 percent to 3.8 percent [12], an 82 percent relative collapse [3]. Copy/paste rose about two thirds in share over the same span [4]. Cloned code carries 15 to 50 percent more defects, because the eventual fix lands on one copy and misses the others [14], and error-masking constructs such as empty catch blocks and bare excepts are up 47 percent, with two-week churn up another 15 percent [13]. The reading is straightforward: you cannot refactor what you have not read, you can only append to it [20].
Agent reviewers are the answer the industry reached for, from CodeRabbit and Bugbot to Copilot review [15], and on the mechanical class they earn their place: naming drift, unhandled branches, resource leaks, a changed signature with three stale callers [16]. On this class they cannot help, because author and reviewer are drawn from overlapping training data, often the same model family, and share a prior about what good code looks like [17]. That shared prior is the thing under audit. It is what produced the timeout assumption in the first place.
Which leaves the phrase everyone has agreed to accept. Google's framing is that AI-generated code is approved by engineers, and "human in the loop" has quietly come to mean a human was present rather than a human understood [18]. The review step survives as an interface element while the code goes in unread [19]. At 2:47 AM the only reviewer property that mattered was knowledge held outside the file, and no amount of reviewer throughput, human or synthetic, supplies it.
What to watch
- Whether agent review products begin asserting service-level invariants such as idempotency keys, instead of only the diff-local properties visible in the file.
- Whether GitClear's moved-code share stops falling in the next reporting period, the first measurable sign that merged code is being read again.
- Whether the next DORA cohort shows stability recovering at high AI adoption, or the throughput-and-instability pairing holds for a third reading.