Build Not yet confirmed elsewhere 1 publisher3 min readPublished Aug 27, 2026, 2:17 PM PDT

Harness choice moved token use 83-fold with the model held constant

The spread traces to two numbers you can read off your own logs: the tokens a harness spends before any work starts, and how many turns it takes. Together they predicted total token use with an R-squared of 0.99.

The Engineer · Build desk

How we use AI Send a correction

Illustration accompanying Harness choice moved token use 83-fold with the model held constant — Generated illustration

What happened

A June benchmark ran twelve harness configurations through OpenRouter on the same twelve Python tasks, so every configuration reached the same API and the same model weights.
Composio's August comparison of eight harnesses on DeepSeek V4 Flash put cost per successful task between $0.028 for Pi Agent and $0.195 for Claude Code.
Artificial Analysis scores 326 tasks with pass rates averaged over three attempts each, drawing on DeepSWE, Terminal-Bench v2.1 and Scale AI's SWE-Atlas-QnA.

Why it matters

decision The per-token rate a procurement team negotiates applies to a token count someone else already fixed, and the person who chose the harness is usually an engineer installing a CLI, not a signatory.
cost Because the prompt floor is billed on every turn, overhead tracks conversation length rather than task difficulty, so trivial tickets carry the same scaffolding bill as hard ones.
capability Two numbers already sitting in your own logs, prompt floor and median turns to done, are enough to forecast agent spend without commissioning a harness bake-off.
constraint One pass per cell will support an argument about mechanism but not a vendor ranking, which pushes anyone writing a contract toward the averaged index instead.

Divide the endpoints and the number gets bigger. 292,000 tokens per solved task for OpenClaw against roughly 3,500 for Aider in architect mode is a factor of about 83 ^[8]^[25], not the 70-fold in the writeup's own headline ^[30]. Round numbers travel better than division does. Either figure sits far outside anything a buyer wins by negotiating price per million tokens.

The floor recurs because of how serving works. Every inference request needs its context, and the relevant history is either supplied again or reconstructed by the serving system, so the provider processes large overlapping blocks of text on each turn, including the harness's system prompt and its tool descriptions ^[4]. The startup tax is that baggage measured once: the system prompt, the tool descriptions and the environment setup, around 700 tokens for Aider in architect mode against around 26,000 for OpenClaw ^[9]. The June author's regression multiplies it by turn count and predicts tokens per solved task with an R-squared of 0.99 across both models ^[10].

You can run that regression backwards. Divide 292,000 by 26,000 and you get about eleven ^[24]. So on the tasks OpenClaw solved, roughly eleven resends of the same scaffolding account for most of the token spend, and the actual task content is what is left over. The fifteen-turn illustration in the post, 390,000 input tokens of scaffolding ^[1], is above the measured per-task total, which is a useful reminder that it was an illustration and not a measurement.

Dollars did not spread as far as tokens. Composio's eight-harness comparison put cost per successful task between $0.028 for Pi Agent and $0.195 for Claude Code, a range of about 7x ^[17]^[27]. Different harnesses, and different work: thirty workflows across Airtable, Gmail, Google Calendar, Google Sheets, GitHub, Slack and PostHog, each under a 900-second ceiling ^[14]. DeepAgents matched Claude Code's pass rate exactly at a quarter of the cost per success, so around five cents ^[18]^[28]. Composio graded with a programmatic verifier rather than an LLM judge, on isolated fixtures seeded with decoys and near-identical keys ^[15], and it disclosed that Pi ran a different reasoning setting across two model providers and that Prime Agent produced only 24 gradable runs out of 30 ^[19]. That is careful benchmarking, and it is also why the result cannot be read as single-variable.

For the token ordering to transfer to your invoice, your turn distribution has to resemble theirs. The June suite was twelve Python tasks with one pass per harness, task and model combination, so it carries no variance estimates while agent runs are stochastic ^[5]^[11]. Artificial Analysis is the better instrument for ranking live pairings, at 326 tasks with pass rates averaged over three attempts, plus a controlled swap that holds Claude Opus 4.7 fixed across Claude Code, Cursor CLI and Opencode ^[20]^[22]^[23]. What transfers without any benchmark is the method: measure your harness's prompt floor, then your median turns to done ^[12].

What to watch

Whether the June author reruns with multiple passes per cell, since Nemotron 3 Ultra's free OpenRouter tier makes replication close to costless.
Whether Artificial Analysis widens its Claude Opus 4.7 controlled swap beyond Claude Code, Cursor CLI and Opencode.
Whether harness projects start publishing their system prompt and tool-description token floor as a documented figure.

Clarity's read

What the record supports and how the coverage leans. The claims behind it follow.

Reality

Evidence58
Adoption34
Hype gap+12
Incentives42
Confidence55

Why these scores

Claim ledger

Ranked by verification strength, evidence, and original report placement.

[1]
The June author noted that a harness carrying a 26,000-token floor through fifteen turns spends roughly 390,000 input tokens on scaffolding alone.
ReportedSupported2 sources— create a free account to open them View cited source
[2]
The June post described the scaffolding overhead as a 40x difference that would be tolerable if paid once, but the resend pattern makes it otherwise.
ReportedSupported2 sources— create a free account to open them View cited source
[3]
Three recent benchmarking efforts suggest the harness, the software that steers a model through tasks, may matter as much as the model when pricing an AI coding agent.
ReportedSupportedView cited source
[4]
Each inference request needs context, and the relevant history must either be supplied again or reconstructed by the serving system, so the provider processes large overlapping blocks of text on every turn, including the harness's system prompt and tool descriptions.
ReportedSupportedView cited source
[5]
In June, an independent benchmark compared 12 configurations across two models on the same 12 Python tasks.
ReportedSupportedView cited source
[6]
The June benchmark ran Aider, Claude Code, Codex, Goose, Hermes, Kilo, Kimi Code, Nanobot, OpenClaw, Opencode and Qwen Code, counting Aider's architect mode separately, with all twelve configurations going through OpenRouter on the same tasks so each harness used the same API and model.
ReportedSupportedView cited source
[7]
The June suite ran on DeepSeek V4 Flash and then on Nvidia's Nemotron 3 Ultra, which offers a free tier on OpenRouter so the run can be repeated at no cost.
ReportedSupportedView cited source
[8]
The June post reported tokens per solved task ranging from roughly 3,500 for Aider in architect mode to 292,000 for OpenClaw, and the ordering barely moved between the two unrelated models.
ReportedSupportedView cited source
[9]
The startup tax is the harness's system prompt, tool descriptions and environment setup shipped before a prompt does any work, reported as around 700 tokens for Aider in architect mode against around 26,000 for OpenClaw.
ReportedSupportedView cited source
[10]
The startup tax multiplied by the turn count predicts tokens per solved task with an R-squared of 0.99 across both models.
ReportedSupportedView cited source
[11]
The June benchmark ran a single pass per harness, task and model combination, so it carries no variance estimates, and agent runs are stochastic.
ReportedSupportedView cited source
[12]
The June author advises developers looking to cut agent spend to look first at the prompt floor and the turn count before touching anything more sophisticated.
ReportedSupportedView cited source
[13]
In August, Composio compared eight harnesses using DeepSeek V4 Flash on 30 enterprise workflows.
ReportedSupportedView cited source
[14]
Composio reported thirty workflows spanning Airtable, Gmail, Google Calendar, Google Sheets, GitHub, Slack and PostHog, with each task running under a 900-second ceiling.
ReportedSupportedView cited source
[15]
Composio graded outcomes with a programmatic verifier rather than an LLM judge, using isolated fixtures seeded with decoys and near-identical keys.
ReportedSupportedView cited source
[16]
Composio reported 240 runs, of which 129 workflows completed successfully.
ReportedSupportedView cited source
[17]
In Composio's comparison, cost per successful task ranged from $0.028 for Pi Agent up to $0.195 for Claude Code.
ReportedSupportedView cited source
[18]
DeepAgents matched Claude Code's pass rate exactly while costing a quarter as much per success.
ReportedSupportedView cited source
[19]
Composio disclosed that Pi ran a different reasoning setting across two model providers and that Prime Agent produced only 24 gradable runs out of 30, caveats that rule out calling the comparison a clean single-variable experiment.
ReportedSupportedView cited source
[20]
Artificial Analysis reported an index combining DeepSWE, Terminal-Bench v2.1 from the Laude Institute and Scale AI's SWE-Atlas-QnA, coming to 326 tasks with pass rates averaged across three attempts each.
ReportedSupportedView cited source
[21]
Artificial Analysis reports cost per task, token use and wall time for each harness-model pairing.
ReportedSupportedView cited source
[22]
Artificial Analysis publishes a controlled comparison that holds Claude Opus 4.7 fixed while swapping between Claude Code, Cursor CLI and Opencode.
ReportedSupportedView cited source
[23]
The source recommends treating the June overhead finding as evidence for the mechanism and the larger Artificial Analysis index as the better instrument for comparing current production-scale pairings.
ReportedSupportedView cited source
[24]
OpenClaw's tokens per solved task divided by its startup tax implies roughly eleven turns of resent scaffolding per solved task.
DerivedSupported2 sources— create a free account to open them
[25]
The reported endpoints of the June token range differ by a factor of about 83.
DerivedSupported1 source— create a free account to open them
[26]
OpenClaw's startup tax is about 37 times Aider architect mode's.
DerivedSupported1 source— create a free account to open them
[27]
Composio's cost per successful task spans about 7x from cheapest to most expensive harness.
DerivedSupported1 source— create a free account to open them
[28]
A quarter of Claude Code's $0.195 per successful task puts DeepAgents near $0.049.
DerivedSupported1 source— create a free account to open them
[29]
About 54 percent of Composio's runs completed successfully.
DerivedSupported1 source— create a free account to open them
[30]
The published headline on the June comparison states that token use varied 70-fold across Aider, Claude Code and OpenClaw running an identical model.
ReportedContested2 sources— create a free account to open them View cited source

Sources

1 independent publisher whose own reporting we read for this story.

thenewstack.io
1 article · August 27, 2026
Aider, Claude Code, and OpenClaw ran an identical model. Token use varied 70-fold.

Ask Clarity

Have a question this piece left open? The answer comes from its reporting and claim ledger — nothing else.

Try

Clarity analysis

How the coverage leans and what the record actually supports. Read the methodology

Perspective Coverage

How much of the coverage speaks to builders, operators and investors. It measures emphasis, not whether the story is right.

Builder

For builders the actionable core is mechanical: the harness resends its system prompt, tool descriptions and environment setup on every turn, so a floor of ~700 tokens (Aider architect mode) versus ~26,000 (OpenClaw) compounds into tokens-per-solved-task of roughly 3,500 versus 292,000 on the same model and tasks. Startup tax times turn count predicted total token use with an R-squared of 0.99 across two unrelated models, and per-turn context growth was similar everywhere, so the lever is the floor and the number of turns rather than clever context management. The caveat matters as much as the finding: one pass per harness-task-model combination, no variance estimates, twelve Python tasks.

Reality

Evidence, adoption, hype gap, incentives and confidence, each read independently from the current record on the scale shown beneath it. How these are measured.

Evidence58: 0100
Convergent benchmark evidence, single-publisher and partly single-pass
Three independent measurement efforts point the same way, methods are described concretely (shared OpenRouter routing, programmatic verifier, 326 tasks averaged over three attempts), and the mechanism has a quantified fit (R-squared 0.99). But the whole cluster rests on one publisher's account, the token-spread benchmark ran a single pass with no variance estimates, the vendor comparison has disclosed control breaks, and the source's own headline figure (70x) conflicts with its reported endpoints (~83x).
Adoption34: 0100
Widely benchmarked tooling, no production usage disclosed
Adoption evidence here is confined to test harnesses being exercised in benchmarks: twelve configurations run through OpenRouter, eight harnesses across thirty enterprise-style workflows, and a continuously tracked index of harness-model pairings. That shows the tools are real, runnable and comparable at benchmark scale, but the sources disclose no deployment counts, customer usage, revenue or production spend for any harness, so real-world uptake is unmeasured.
Hype gap+12: −100 understated0 aligned+100 overstated
Mechanism modestly oversold by the framing, not by the numbers
The underlying measurements are reported carefully and the article volunteers its own limits (single pass, stochastic runs, vendor caveats, index preferred for production comparison). The overstatement is at the framing layer: a headline spread figure (70-fold) that does not match the body's endpoints (~83x), extrapolation from twelve Python tasks and thirty fixtures toward general agent pricing, and no accompanying task-quality comparison to show cheap harnesses solve equivalent work.
Incentives42: 0100
One vendor-run comparison, disclosed; other instruments more arm's length
Composio published the August comparison it is cited for, which is a self-interest exposure, though it disclosed the control breaks (Pi's differing reasoning setting, Prime Agent's 24 gradable runs of 30) transparently. The June benchmark is described as independent and rerunnable at no cost on a free Nemotron 3 Ultra tier, and Artificial Analysis is a third-party index — both reduce distortion risk. No pricing, sponsorship or commercial relationships between the publisher and any named harness vendor are disclosed either way, so residual uncertainty stays moderate.
Confidence55: 0100
Moderate: mechanism credible, magnitudes provisional
The causal story — resent scaffolding floors times turn count driving token spend — is coherent, quantified and reproduced in ordering across two unrelated models, so confidence in direction is fair. Confidence in the specific magnitudes is lower: one publisher, one single-pass token benchmark, a vendor comparison with disclosed caveats, an internal headline/body discrepancy, and no independent replication or vendor rebuttal in the cluster.

Topics and entities

Follow any of these and your For You feed starts watching them — no settings page required.

Topics

Benchmark Methodology and Reproducibility Follow
Agent harness token economicsFollow
LLM inference cost managementFollow
AI Coding Agents Follow
Prompt caching and serving pathsFollow