Skip to content

Build Not yet confirmed elsewhere1 publisher 3 min read Published

Mutation testing pinned the rounding logic, but a separate 3p VAT bug slipped past 202 green tests

A .NET invoicing library built with Claude Code came with property tests, hand-calculated golden scenarios and green CI on three operating systems, all of it blind to the half-penny inputs where half-up and banker's rounding diverge.

The Engineer · Build desk

How we use AI Send a correction

Illustration accompanying Mutation testing pinned the rounding logic, but a separate 3p VAT bug slipped past 202 green tests
Generated illustration

What happened

  • A .NET invoicing library written with Claude Code over about a week reached its third phase with 202 tests passing on two target frameworks and green CI on three operating systems.
  • That same build would have been wrong by 3p on any UK invoice containing three items at 3.99 pounds, a price point the author calls the most common in British retail.
  • Switching the rounding mode to ToEven failed five tests, every one of them a unit test of the rounding helper, while all twenty-five hand-calculated golden invoice scenarios stayed green.
  • Stryker.NET fell over on the multi-target setup, so after fifteen minutes the author abandoned it and applied mutations by hand, choosing which behaviours to attack instead of taking a score.
  • Every step of the release pipeline reported green after a v0.3.0 tag packed version 0.2.0 and NuGet's 409 was absorbed by --skip-duplicate, leaving nothing published.

Why it matters

  • constraint A suite the agent produced alongside the implementation cannot serve as the acceptance gate for that implementation, because both halves encode the same reading of the requirement, which means the gate has to come from somewhere else.
  • decision Teams using agent-written tests have to decide explicitly who chooses the fixture values, because inputs that never land on a boundary will pass whichever policy the code happens to have implemented.
  • cost The substitute for a coverage percentage is human attention spent one mutation at a time, and that is the price of knowing whether four specific rounding call sites are pinned.
  • exposure The penny lands on whoever files the return and on the counterparty reconciling line-level tax, and since both rounding bases are permitted, neither side is obviously in the wrong when the totals differ.

Rounding policy is only observable at a boundary. Half-up and banker's rounding return the same value for every input except one that lands exactly on a half unit. A fixture set with no half-unit inputs therefore cannot tell you which policy the code implements.

The library rounds half-up, MidpointRounding.AwayFromZero, because that is what tax authorities expect, against a .NET default of banker's rounding, according to the write-up on dev.to [5]. The suite had property tests and twenty-five golden scenarios with hand-calculated values across USD, JPY and KWD [20]. None of those scenarios contained a midpoint value [7]. Flipping the mode left 197 of the 202 tests entirely unmoved [23].

The scenario that looks like it was written for this case is 995 yen at 10 percent, which yields 99.5. ToEven takes that to 100, because 100 is the even neighbour, so both policies agree and the test discriminates nothing [8].

The 3p figure has the same shape. Three lines at 3.99 is 11.97 [21], and 3p spread over three identical lines is a penny each [22]. The author does not show the working, so read the per-line split as inference: for the error to scale with the item count, each line has to land on a half penny and get pushed the wrong way. That is precisely the input class the fixtures lacked [7].

His own stopping rule did not save him either. Fewer than three failures on an arithmetic mutation counted as a hole [15], and the ToEven mutation returned five, which clears that bar [24]. All five sat in the rounding helper's own unit tests [6]. Five failures cleared the numeric bar, but their concentration in a single file meant the call sites elsewhere were still unpinned. When every failure lives in one file, the coverage stops at that function's boundary rather than extending to the code that calls it.

The discount mutation is a cleaner reading of the same fault. Pull rounding out of the invoice-level discount calculation and exactly one test fails, a generic property test asserting that all decimals get rounded, because every discount scenario used a clean percentage of a clean subtotal [9].

The culture test is the one I would frame and hang on the wall. It swaps CurrentCulture to de-DE and tr-TR, exports CSV, asserts [10]. Nothing wrong with it. Then swap InvariantCulture for CurrentCulture in the export code and it still passes, because the fixture invoice totals 100, 20 and 120, and 100m.ToString() under de-DE gives "100" [11]. Change a unit price to 10.50 and the same mutation kills three tests [12]. The fixture's invoice totals were whole numbers, so there was no decimal separator to get wrong, no matter how sound the assertion was.

This is one library's suite, so 202 is not a number about your code. The transferable part is the condition. If the same process wrote the implementation and chose the inputs, and no input sits on a boundary where the implementation had to pick a policy, then the suite documents behaviour rather than constraining it. That is the author's own framing: the agent writes both halves, they pass, and what you have confirmed is that the code agrees with itself [17].

The domain is not incidental to the bill. HMRC and the ATO both permit tax to be rounded per line instead of on the subtotal, and when the accounting system at the other end rounds per line, the totals disagree by a penny [25].

The change that actually altered the outcome was procedural. By the third phase the author had the agent answer design questions on paper first, with real numbers, before code [18]. That puts an expected value outside the implementation, which is the only place an oracle can live.

What to watch

  • Whether Stryker.NET handles the multi-target setup well enough to make this loop cheap, since hand-picked mutations do not scale past a few call sites.
  • Whether the per-line tax mode ships, and what the paper-first design answer produced for the subtotal-versus-line disagreement.
  • Whether the release workflow gains a step that fails on a NuGet 409 instead of letting --skip-duplicate absorb it.

Clarity's read

What the record supports and how the coverage leans. The claims behind it follow.

Reality

Evidence48
Adoption10
Hype gap+12
Incentives28
Confidence55
Why these scores

Claim ledger

Ranked by verification strength, evidence, and original report placement.

  1. [1]

    The author built a .NET invoicing library with Claude Code over about a week, with zero dependencies, VAT and GST arithmetic, and JSON and CSV export.

    Reported Supported Source: dev.to write-up by the library's authorView cited source
  2. [2]

    By the third phase the project had 202 tests passing on two target frameworks and green CI on three operating systems.

    Reported SupportedView cited source
  3. [3]

    The library would have been wrong by 3p on any UK invoice with three items at 3.99 pounds.

    Reported SupportedView cited source

Get this ledger inside ChatGPT. Ask about any of our stories and the answer brings its figure, each claim’s verdict and the sources. Free, no account.

Add to ChatGPTAdd to Claude

Sources

1 independent publisher whose own reporting we read for this story.

  1. dev.to

    1 article · August 30, 2026

    My tests were describing the code, not checking it

Share your take

Let Clarity write the post for you.

Signed-in readers get a short post drafted on this story in the register they choose — narrative, analytical, or a direct position — editable to the last word before it goes anywhere. The share buttons at the top of this story work without an account.

Topics and entities

Follow any of these and your For You feed starts watching them — no settings page required.

Topics

Loading related stories