BuildNot yet confirmed elsewhere1 publisher2 min readPublished
Coding agents add mocks in 36% of test commits where humans add them in 26%
Hora and Robbes's MSR 2026 study of 1.2 million commits found coding agents add mocks in 36% of test commits, against 26% for humans. A dev.to post turns that gap into an argument for written mocking policy.
The Engineer · Build desk

Key figures: agent test commits add mocks about 1.4 times as often as human test commits. The study isolated 48,563 commits authored by coding agents, about 4% of the 1.2 million commits measured across public repositories.
| Figure | Value | Note | Claim |
|---|---|---|---|
| Agent mock-adding rate vs human test commits, about | 1.4 times | 7 | |
| Commits by coding agents | 48,563 commits | 2 | |
| Agent-authored, about | 4% of commits | 8 | |
| Public-repo commits measured | 1.2 million commits | 1 |
What happened
- Coding agents authored 48,563 of the commits, about 4% of the sample, and the study compared their test commits with everyone else's.
- Agent commits cluster on a single test-double style, while human commits spread across stubs, fakes, spies and mocks.
- According to the post, agent-generated suites also come out more brittle than the human baseline on the measures the paper reports.
Why it matters
- decision A team that lets agents write tests has to choose whether to write down which test doubles are allowed, or leave the existing suite as the unwritten rule.
- exposure Mock-heavy tests merged today become examples the next agent session copies, so repos with a long mocking history have the most patterns to amplify.
- constraint The 1.4x ratio comes from public repositories, so it is a prompt to measure your own agent and human commits, not a forecast for your suite.
The gap is 10 percentage points, and the post's author is careful about what that licenses. The author wrote: "Ten percentage points on a measured dataset is a clear signal; it is not evidence that agents mock everything in sight." [6] As a ratio, agent test commits add mocks about 1.4 times as often as human ones [7].
The sample limits how far that ratio travels. Agents wrote about 4% of the 1.2 million commits [8]. The post does not say how many of those are test commits or which languages and agents the set covers. For the 36% to hold in your repo, your agents, your language and your mocking history would have to look like the public sample [3].
The causal story is the author's. The paper, as the post describes it, measured rates of test-double use and brittleness [1][9]. The remedy is the author's inference: the fix lives at the convention layer, because review cannot keep pace [12]. The mechanism offered is that the codebase is the specification. "Agents mock what the codebase mocks," the author wrote [10].
The case against review rests on an illustration. A reviewer who catches eight mocks in a diff of twenty tests has shipped twelve, and those twelve become examples the next session samples [11]. The numbers are tidy, as hypotheticals tend to be.
In our view, the remedy is cheap enough to try before the evidence is complete. The first step is to measure: the share of test commits that add a test double, split by agent and human authors. That is the comparison the paper made [3][4]. If your own gap comes out near zero, a policy can wait. If your suite is already mock-heavy and agents write tests at volume, we think the post's case holds, and a written convention gives the reviewer something to enforce.
What to watch
- The paper itself (arxiv 2602.00409): its brittleness measures, languages and agent set decide how far the 36% travels.
- Any team that publishes before-and-after mock rates for agent commits once a written convention is in place, since that would test the post's remedy.
Clarity's read
What the record supports and how the coverage leans. The claims behind it follow.
Reality
- Evidence45
- Adoption
- Insufficient
- Hype gap+25
- Incentives
- Insufficient
- Confidence55
Claim ledger
Ranked by verification strength, evidence, and original report placement.
- [1]
A 2026 paper by Hora and Robbes (arxiv 2602.00409, MSR 2026) measured 1.2 million commits across public repositories.
- [2]
The study isolated 48,563 commits authored by coding agents.
- [3]
Agent test commits add mocks 36% of the time.
- [4]
Non-agent test commits add mocks 26% of the time.
- [5]
Agents concentrate on a single test-double style, where humans spread across stubs, fakes, spies, and mocks.
- [6]
"Ten percentage points on a measured dataset is a clear signal; it is not evidence that agents mock everything in sight."
- [7]
Agent test commits add mocks about 1.4 times as often as human test commits.
- [8]
Agent-authored commits are about 4% of the 1.2 million commits measured.
- [9]
According to the post, the resulting agent-generated suites are more brittle than the human baseline on the measures the paper reports.
ReportedInsufficientSource: dev.to post by tmfrisinger, originally published at tddbuddy.comView cited source - [10]
"Agents mock what the codebase mocks."
- [11]
The post's illustration: a reviewer who catches eight mocks in a diff of twenty tests has caught eight and shipped twelve, and the twelve ship with precedent weight, since each becomes an example the next session samples.
- [12]
The post's author argues the fix lives at the convention layer, which shapes what the coding agent sees, because review cannot scale to the rate at which agents write tests.
Sources
1 independent publisher whose own reporting we read for this story.
- dev.toOver-Mocking Is a Governance Problem, Not a Review Smell
1 article · October 12, 2026
Topics and entities
Follow any of these and your For You feed starts watching them — no settings page required.
Topics
- Mining software repositoriesFollow
- AI Coding AgentsFollow
- Test doubles and mockingFollow
- Software Testing PracticeFollow