Build1 publisher3 min readPublished
A four-rule refund check runs $14,600 a year through the cheapest Claude model
A dev.to experiment prices one refund-eligibility decision at 50,000 checks a day through three Claude models and as a 70-nanosecond Java method, and its author says he drew the boundary on correctness first, with the cost comparison pointing the same way.
The Engineer · Build desk
What happened
- A dev.to experiment costs one refund-eligibility check at an assumed 50,000 calls a day, putting Claude Opus 5 at $73,000 a year, Sonnet 5 at $29,200 and Haiku 4.5 at $14,600.
- Halving that prompt estimate still leaves the cheapest model at $7,300 a year, and the post says no token count makes the comparison close.
- Three conditions are listed as capable of overturning the comparison, in rough order of likelihood, with a collapse in call volume first and a thousandfold fall in prices last.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- decision The per-call price is identical either side of the boundary, so the sizing question for an agent component is how often it runs.
- cost Retries are billed on the model path, and eligibility is re-checked on every refund request, every status enquiry mentioning a return and every retry, so the bill grows with failure rates the team does not control.
- contradiction The cost table is the loudest part of the post while its author says cost was not why he drew the boundary, so a team that takes the number without the testability argument has nothing left once volume drops.
Each model row is a published per-token price times one estimated prompt: 500 tokens in, 60 out, with the refund policy carried as a system prompt on every call [10]. That is 560 tokens a check, so the Haiku 4.5 row implies a blended $1.43 per million tokens [23]. The post builds the estimate from list prices and does not mention prompt caching or batch tiers [22].
At the assumed volume, the same estimate implies about 28 million tokens a day, or roughly 320 tokens a second sustained [30]. Billing is one limit at that rate, and rate limits are another.
RefundEligibility.evaluate was measured in a warmed-up loop at about 70 nanoseconds and costed as rented CPU time at $0.036 per vCPU-hour [8][9]. That hourly rate is $1e-5 a second, and 70 nanoseconds of it is $7.0e-13, which is the per-call figure in the table [26]. It counts CPU time only. The JVM and the host are paid for already, so the marginal cost of that call on a box you rent anyway is nearer zero than the $0.000013 a year printed [9].
Haiku against the method is about 1.1 billion to one, Opus about 5.6 billion [24][25], so the post's "about a billion to one" is the conservative end of its own table [11].
The check itself is four booleans: delivered, paid, inside the return window, belongs to the customer [2]. The post's decision flow sends work that runs once per conversation to the model, intent and retrieval, and work that runs per request, per retry, per rule to software, meaning policy, eligibility, risk tiers and scoping [14]. The author calls $0.004 a bargain for turning "the shoes don't fit, can I send them back?" into a structured request, and says no rules engine he would want to maintain does it as well [15].
The components that ended up in the domain layer are the high-frequency ones with a right answer, and that boundary was drawn as a correctness decision first, with the cost comparison pointing the same way [16]. "Cost isn't the reason for the boundary. It's just the easiest reason to put on a slide," the author wrote [20]. Even at zero cost he would keep eligibility in a method, because "it's testable, it's inspectable in an audit, and it can't have a bad day" [21].
The financial half depends entirely on volume. At 500 checks a day the post puts the model path at $2, which is 500 calls at the Opus per-call price and $730 across a year [17][27]. Holding the Haiku path under $1,000 a year needs volume down near 3,400 checks a day [28]. The second condition on the list changes the design: add goodwill exceptions and "use your discretion for loyal customers" and there is nothing left to express as four booleans [18].
What to watch
- Whether a cached policy prompt or a batch tier lands under the implied $1.43 per million blended tokens, which would move the model rows without touching the code row.
- Whether later parts of the experiment publish measured production volume instead of the 50,000-a-day assumption.
- Whether the repo's PromptSize value gets re-costed once the refund policy acquires discretion clauses.