Leadership1 publisher2 min readPublished
Anthropic locates the moment an agent team outgrows manual testing
Its engineering post argues that automated tests for multi-turn tool use belong in place before an agent scales. The evidence behind the advice is three deployments, one of them Anthropic's own.
The Board Room · Leadership desk
What happened
- Anthropic published engineering guidance on evaluating AI agents, defining an eval as an input plus grading logic and focusing on automated tests that run during development without real users.
- Without evals, the post says, teams cannot separate real regressions from noise, test a change against hundreds of scenarios before shipping, or measure whether the agent improved.
- Descript, cited as a customer example, now runs two separate eval suites, one for quality benchmarking and one for regression testing of its video-editing agent.
- Bolt AI's system runs its agent and grades the output with static analysis, browser agents that test the resulting apps, and LLM judges for behaviors such as instruction following.
Compiled by The Board RoomSomething wrong?How this is made
Why it matters
- decision An engineering leader is choosing between paying for suites before the agent scales and paying later, when the post says evals themselves become the bottleneck for improving it.
- exposure Both customer systems grade with model calls, so part of any eval budget arrives at a model vendor as inference spend, and the vendor here is the one recommending the coverage.
- constraint A suite can mark a better answer wrong, as the tau2-bench case shows, so red scores need a human read and the graders need revising as models change.
- cost Anyone building the business case works from one datapoint on duration and no cost figure, so the spend has to be argued from avoided rework rather than a published benchmark.
Anthropic's post gives two sequencing answers and does not rank them. Teams can get surprisingly far on manual testing, dogfooding and intuition, and rigorous evaluation can look like overhead that slows shipping [4]. Evals are especially useful at the start of agent development, where they encode expected behavior and force a product team to say what success means [14]. The same post describes teams that add them later, at scale, once evals have become the bottleneck for improving the agent [15].
The guidance draws on named cases [17]. Claude Code began with fast iteration on feedback from Anthropic employees and outside users, then got evals: first narrow ones for concision and file edits, later harder ones for behaviors like over-engineering [8]. Descript moved from manual grading to LLM graders whose criteria the product team defined, with periodic human calibration [10]. Bolt AI is the only case with a clock on it, three months to build its system after the agent was already widely used [12]. Anthropic did not publish what any of that cost in engineer time.
The post is Anthropic's own, and both customer systems it cites grade output with model calls: LLM graders at Descript [10], LLM judges for instruction following at Bolt AI [13]. More eval coverage means more inference. The failure it describes is specific enough for a leader to look for in their own bug queue, though. The breaking point often comes, Anthropic wrote, when users report the agent feels worse after changes and the team has no way to check that beyond guessing and testing by hand [5].
Agents work across many turns, calling tools and modifying state, so an early mistake propagates and compounds [2]. That makes a static test both necessary and fallible, and the post supplies the fallible half. Opus 4.5 solved a tau2-bench flight booking problem by discovering a loophole in the policy; it "failed" the evaluation as written, but in Anthropic's account came up with a better solution for the user [7].
Whoever writes the grading criteria decides what "do it well" means in production, and Descript let the product team define them [10]. Its suites are split accordingly: one for quality benchmarking, one for regression testing, graded against three dimensions the team set out as don't break things, do what I asked, and do it well [11][9].
What to watch
- Whether Anthropic or its customers publish engineer-time or cost figures for building an eval suite.
- Whether any of the three named teams reports a specific regression its suite caught before shipping.
- Whether more cases like the Opus 4.5 tau2-bench loophole appear, forcing teams to budget for revising graders as models improve.