Leadership1 publisher2 min readPublished
A senior reviewer's hours set the ceiling on how much agent-written code a team can absorb
Addy Osmani argues that teams leaning on agentic coding accrue a debt in code nobody can explain. His evidence is one 52-person trial and a student team, and what it supports is a question about review staffing.
The Board Room · Leadership desk
What happened
- Addy Osmani's post names comprehension debt as the widening gap between how much code exists in a system and how much of it any human genuinely understands, and says velocity metrics do not show it.
- Margaret-Anne Storey describes a student team that by week seven could not make simple changes without breaking something, because nobody could explain why the design decisions had been made.
- Osmani argues the review economics have inverted: seniors once read faster than juniors could write, and now a junior generates faster than a senior can audit, turning a quality gate into a throughput problem.
- He adds that deterministic checks have a hard ceiling, since a suite covering all observable behaviour would often be more complex than the code it validates.
Compiled by The Board RoomSomething wrong?How this is made
Why it matters
- constraint The input a team can actually buy is senior reading hours, and adding generation capacity without them lengthens the queue while leaving the limit where it was.
- exposure Because nothing degrades visibly first, the manager who owns the system learns about the gap when a change stalls or an incident lands, with no lead time to staff for it.
- cost The capability being eroded is debugging, so the bill falls later on whoever is expected to diagnose a system they never built a theory of.
- decision Whether engineers delegate passively or interrogate the model depends on how much reading time the schedule allows, so it is settled in staffing and review policy.
Reading a colleague's pull request was never only a defect check. Osmani wrote that it forces comprehension and distributes knowledge about what the codebase actually does across the people responsible for maintaining it [9]. A test suite cannot buy that back. A review skimmed to keep the queue moving still merges the code, and the understanding it was supposed to spread stays put.
The signals that used to justify a fast merge are the ones agentic output produces most reliably. Osmani wrote that the code is syntactically clean, often well-formatted and superficially correct, and that surface correctness is not systemic correctness [10]. Technical debt announces itself through slow builds and tangled dependencies. This one shows a clean codebase and green tests, and by Osmani's account the cost surfaces later [3].
The trial carrying most of the empirical weight is small, and the headline figure is quoted loosely. Anthropic's "How AI Impacts Skill Formation" randomised 52 engineers learning a new library; the assisted group finished in roughly the same time as the control and scored 50 percent on a follow-up comprehension quiz against the control's 67 percent [4]. Osmani describes that as 17 percent lower. Subtract, and it is 17 percentage points. Divide 17 by 67, and the assisted group scored about a quarter lower in relative terms [15]. Osmani does not say how the arms were split; divided evenly they would hold about 26 engineers each [16]. The steepest declines were in debugging, with smaller drops in conceptual understanding and code reading [5].
Beyond that trial the case is anecdotal: Storey's account of one student team, and Osmani's reading of Hacker News threads where engineers were working out what rigour looks like once the bottleneck has moved [13]. The post does not measure comprehension debt across a production codebase. Osmani names the paper's arXiv listing, 2601.20245, for anyone who wants the numbers themselves [7].
One finding points at a lever a manager holds. The Anthropic researchers emphasise that passive delegation, the "just make it work" pattern, impairs skill development far more than active, question-driven use [6]. In my view which of those two a team gets is decided by how much time the schedule leaves for reading, and that is set in headcount and sprint planning. Osmani wrote that an engineer had put the bottleneck as a competent developer understanding the project, and added: "AI doesn't change that constraint. It creates the illusion you've escaped it." [14]
What to watch
- Whether the arXiv paper at 2601.20245 reports arm sizes, effect sizes and intervals that hold up the 50 against 67 result.
- A replication with working engineers on an existing production codebase instead of a new library.
- Any team publishing review hours per merged change alongside its velocity numbers. That pairing would make the gap measurable.