Databricks finds the harness swings coding-agent cost more than the model does

Databricks says its developers often eyeball token costs to work out how expensive a model will be ^[9]. On its own tasks, that habit broke. Sonnet 5 is about 1.7x cheaper per token than Opus 4.8 ^[10], and it consumed 1.9x more tokens getting to an answer ^[11]. Divide one into the other and you would predict Sonnet finishing roughly 12 percent more expensive per task ^[1]. Measured, it came in 7.7 percent higher, $2.09 against $1.94, with task completion six points lower at 81 percent to 87 percent ^[11][2].

Running one model at one thinking effort through Claude Code or Codex against Pi moved cost per task by more than 2x in some cases, with quality unchanged, according to the post ^[12]. Pi sent about 3x less context per turn, kept a tighter working set, and finished in fewer runs ^[13]. The spread between the two top-tier models Databricks names is 1.52x: Opus at $1.94 and GLM 5.2 at $1.28, or 34 percent less per task ^[7][3], a narrower gap than the harness produced ^[4].

Inside Databricks the consequence is a routing policy. The post says its default models were always the most expensive ones, and that this analysis convinced it to push more work to the Haiku and GPT 5.4 Mini class ^[5]. The work it puts in that lane is flipping a flag and updating configs; deeper design exploration stays with the expensive tier ^[6]. GLM, which tied Opus on quality, gets promoted to what Databricks calls a daily driver for coding ^[8]. Someone has to own that routing table and keep it current as versions ship.

The dollar figures themselves are not a yardstick another buyer can hold up to a vendor. Databricks says public benchmarks like SWE-Bench and TerminalBench are useful but cannot answer the questions it has ^[15], and it says task shape and complexity may be different in different contexts ^[17]. It also declined to treat its own point spreads as precise, noting that a couple of points can even out on real work ^[18]. Its own caveat reads: "This isn't meant to be comprehensive, but the exercise surfaced insights that have already made our engineering team meaningfully more efficient with coding agents" ^[16].

The measurement is the part another team can copy: tasks its own engineers actually closed last month, candidates graded against the solutions those engineers shipped, and the result priced per completed task instead of per million tokens. The variable to move next is the harness, with the model and the thinking effort held fixed, because the harness is where Databricks measured the 2x spread at constant quality ^[12]. The 2x2 is model tier against harness context discipline: frontier or cheap on one side, tight or loose working set on the other. Databricks put routine operational work in the cheap-model cell and kept design exploration in the frontier one ^[5][6].