Forty-four percentage points separate what METR's participants predicted from what measurement recorded: they expected to be roughly 25% faster and were roughly 19% slower [4][5][13]. Either end of that is interesting, but the width is the part a budget owner should hold onto. Sixteen experienced developers, working in codebases they already knew, misjudged their own throughput by that margin and were still misjudging it after the work was done [3][4]. Any productivity figure whose underlying instrument is how the work felt inherits an error bar wide enough to swallow the result.
That is the case for reading the spread in enterprise claims as an artefact of instruments rather than a genuine range of outcomes. Convert the two figures cited by Laks Alagappan, a Genpact VP who leads delivery for Tier 1 banks in the UK and Europe, into a common unit: a third off elapsed development time is about 1.5 times the prior rate, while doubled output is 2.0 times [1][2][14]. Two firms can both be reporting in good faith and land that far apart, because one is timing a coding task and the other is counting work that reached production.
The mechanism behind the disappointment is that relieving one stage does not relieve the system. Google's 2025 DORA report associated AI adoption with higher delivery throughput and lower delivery stability [7]. Alagappan reports that banking technology leaders see individual productivity rise and dashboards improve while release cadence barely moves, because change approvals, regression testing, compliance reviews, environment provisioning and operational readiness still set the pace [9]. Once code generation stops being the binding constraint, the queue forms at whatever comes next [12].
The strongest objection to this reading is that sixteen open-source developers on 2025 tooling is not an evidence base for a capex decision, and that METR's own 2026 follow-up suggested newer tools were likely delivering larger gains, with selection effects leaving the size unreliable [6]. That objection is right on its own terms: the 19% figure should not be used as an enterprise baseline. The durable finding is narrower. Self-assessment did not track measured output, which disqualifies the developer survey as a primary instrument whatever the true effect from current tools turns out to be, and on that size the record says we do not know yet.
The decision available this quarter is which instrument to fund, not which tool to buy. Coding speed, lines of code, suggestion acceptance rates and individual output are cheap to collect and, in Alagappan's reading, are not measures of engineering productivity at all [8]. DORA's observation that AI's benefits scale with the maturity of the surrounding system points spend toward automated testing and fast feedback loops [10], and the pre-code work Alagappan flags, including de-conflicting requirements, assessing regulatory impact, mapping application dependencies and investigating production incidents, is where he expects the larger return while being harder to put on a dashboard [11]. The ordering consequence is concrete: a team that starts reporting delivery outcomes now will be able to defend or kill this spend a year from now, and a team reporting acceptance rates will be asked why cadence did not change and will have collected nothing that answers it.
