Skip to content

Build1 publisher3 min readPublished

Invoked in three runs, executed in none: the cost rule that never got asked

A single deep-research request burned 194% of a rolling quota, most of it inside six minutes. The tiering rule that should have caught it never reached a decision point.

The Engineer · Build desk

Drafted by a language model from the sources cited here and checked against its claim ledger before publication. How we use AISend a correction

Photograph accompanying Invoked in three runs, executed in none: the cost rule that never got asked
Photo: politifact.com

What happened

  • A deep-research request to the top-tier model tore through three consecutive five-hour usage windows, the rolling quota Claude enforces before a session has to stop and reset, to answer a single question.
  • The three windows added up to 194%, nearly two entire windows.
  • The first two of the three usage windows were gone inside six minutes.
  • Nobody had done anything unusual; someone had just asked the model to look something up properly.
  • The routing policy the machine runs on has a rule that work which is one step (search, fetch a source, extract a claim) and objectively checkable belongs on the cheapest tier that can do it, not on the model that costs the most per token.

Compiled by The EngineerSomething wrong?How this is made

Why it matters

An account published on dev.to describes one deep-research request to a top-tier model tearing through three consecutive five-hour usage windows to answer a single question [1]. The total came to 194% of the rolling quota, close to two entire windows, and the first two were exhausted inside six minutes [2][3]. Nothing exotic was asked for. Per the same account, nobody had done anything unusual; someone had asked the model to look something up properly [4]. The routing policy on that machine already had a rule for the shape of work involved: single-step, objectively checkable tasks such as search, fetch a source, extract a claim belong on the cheapest tier that can do them, not on the model with the highest per-token cost [5]. Deep research is built entirely out of that shape, feeding a synthesis step that does need judgment [6]. The rule existed. It never had a chance to apply [7]. The reason is structural rather than stylistic. A routing policy engages only at a decision point, the moment a session has to choose which tier does the next piece of work, and most of the policy machinery assumes such a moment exists [8]. Decomposition manufactures those moments: a plan becomes tasks, and each task is a place to ask who does this [9]. A direct "deep-research this" is answered as one thing by whichever model receives it, with no task list to route, so search, fetch, extract and verification all happen inside a single reply from the most expensive available model [10]. That is the whole failure. The tiering logic was never wrong; it was never consulted. There are two ways a spend control dies, and this project hit both. Independently, the author found a third-party tool doing something similar whose budget cap, meant to prevent exactly this kind of burn, had its own bug [11]. A cap that misfires and a cap that never gets reached produce the same invoice. The verification attempt is the useful part. A sibling project ran six blind subagents, no shared context, one prompt each: three ways of asking for the same research crossed with two model tiers, with the topic held constant at the enforcement status of the EU AI Act so wording was the only deliberate variable [12][13]. Three wordings by two tiers is one run per cell, so no result here is replicated [1]. The intended measurement failed outright: a subagent's transcript file stays zero bytes after the run finishes, so nobody could read off which tools had been called [14]. The fallback was self-report, cross-checked against a tool-call count the harness reports and the agent has no hand in writing [15]. The harness fired in half the runs, which looked like a pass [16]. Every triggered run was instructed to call Workflow, the dispatcher that fans research out across sub-agents [17], and Workflow is not available inside a delegated subagent [18]. Each of those runs discovered that mid-task and quietly did the research by hand with the tools it had; nothing crashed, nothing logged an error, and the answers looked normal [19]. One subject improvised its own fan-out, reported its agents converging as if that settled the question, then corrected itself when asked what the agreement was worth: "The convergence I reported between agents is corroboration, not adversarial verification, five agents drawing on an overlapping source pool can agree and still be wrong together" [20]. Catching agreement that is not evidence is the entire job of the step that got skipped [21]. A compliance check testing whether the harness was invoked would have scored that run as passing [22]: invoked in three of six runs, executed in none [23], a 50% measured pass rate against a true rate of zero [2]. Three things to watch. Whether the zero-byte transcript problem gets fixed, because self-report plus a tool-call count is the floor of instrumentation, not the target [14][15].

Loading claim ledger
Loading source directory links
Loading share composer
Loading topic controls
Loading related stories