Auditors reproduced 45-plus billing bugs across 110 AI usage tools, most often retries counted as new work

The retry case is a counting rule. A chain of 429, timeout, then success is one logical operation run three times ^[1]. If the meter keys on executions, the customer is billed three times for one result ^[1]. Every record in that chain is correct on its own, so a dashboard built from the same records stays green ^[2]. The auditors call it the single most common finding ^[2].

Lines re-emitted after a crash look like new usage, much as a retry does, and byte-identical duplicates inside session files turned up in more tools than the auditors expected ^[3]. They say the fix is to dedupe on record id before counting anything, and that its dullness is why it goes undone ^[3].

In some tools, cached-read tokens ride the same field as fresh tokens and are priced the same ^[4]. At cache-hit rates typical for agent workloads, the authors say, that is not a rounding error ^[4].

The grain problem is subtler, and the post's example shows it: a median of 2.5 steps per operation counted over successful attempts, against 4.0 counted over total consumption including retries ^[5]. If both describe one workload, the second is 1.6 times the first ^[11]. The authors say both are defensible and only one matches what the bill should say ^[5].

In the worst of the five classes, a boolean "consumed" field reused for "not set" turned 9,107 referenced observations into a confident zero ^[6]. The audit's answer is a state for not knowing. When an export lacks the field that decides a metric, the finding is marked UNPROVABLE and left unfilled ^[7]. The authors wrote: "Most metering disputes we've seen are not dishonesty; they're someone's UNPROVABLE rendered as a confident number." ^[8]

How far these numbers travel depends on what was tested. Each bug was reproduced with the tool's own parser on synthetic fixtures ^[12]. That shows a parser mishandles a shape. How often the shape occurs in production, or what it cost anyone, the fixtures cannot say, and "45+" counts bugs, not affected tools ^[13]. For the retry result to transfer to your invoice, your runtime has to retry the way the fixture does and your vendor's meter has to count executions.

The free checker works on files on your machine. It scans Codex and Claude Code session logs locally, in about thirty seconds, with nothing uploaded and no account ^[9]. In our view, a clean scan says your logs are internally consistent. Whether the vendor's meter agrees with them is a separate comparison. The figures are the authors' own, as of 2026-10-14 ^[13].