Invest1 publisher3 min readPublished
Amodei commits Anthropic to the only step of his slowdown plan he can enforce alone
Anthropic is seating independent evaluators inside the company immediately, and the evaluators keep control of what they publish. The two steps of Amodei's plan that would slow capability gains need rivals and governments to agree.
The Investor · Invest desk
What happened
- Dario Amodei published an essay on Saturday setting out a plan he calls "pacing the frontier," arguing that the industry must slow the rate at which it improves AI model capabilities.
- The plan has three steps: permanent evaluator access at every frontier company, common safety standards among firms in democratic countries, and government talks with authoritarian states beginning with a bioweapons ban.
- Anthropic safety lead Evan Hubinger endorsed the resignation post and put his own estimate of AI killing all humans above 10 percent within the next decade.
Compiled by The InvestorSomething wrong?How this is made
Why it matters
- precedent One frontier lab has now conceded continuous access at employee level, with no vendor sign-off before publication, so enterprise buyers and legislators can ask for it as a baseline contract term.
- constraint The two steps that would slow capability gains cannot be started by Anthropic alone, so the pace of the frontier stays where competitors and governments set it.
- exposure Incidents that previously came to light only when outside researchers dug them up now have a route out of Anthropic that bypasses its communications function.
- contradiction Amodei asks the industry to slow down while current and former staff describe his own company racing, and former workers attribute the strain to OpenAI's competitive pressure.
The publication right is the part a lawyer can copy. Anthropic says the embedded evaluators will have the same access as its own risk-assessment teams, plus the right to publish their findings without the company's editorial control [4]. Access at employee level, and publication rights the evaluators hold outright, are two lines in a contract schedule, and neither of them slows a training run.
The parts of the plan that would slow one are steps two and three. Step two asks companies in democratic countries to agree on common safety standards that limit the rate of unchecked progress [2]. Step three asks democratic governments to attempt coordination with authoritarian states, starting with agreements that are in everyone's interest, such as a ban on using AI to develop biological weapons [2]. Anthropic committed to step one on its own, with immediate effect [3]. One of the three is in force. The other two, the ones that touch the rate of capability gain, depend on signatures from outside the company [14].
Amodei gave two reasons for urgency: models are increasingly able to build their successors, and the industry, Anthropic included, has had a string of safety incidents [5]. "We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain," he wrote [6]. Fortune's account leaves the evaluators unnamed and their number unstated [15].
The essay landed in the same week that Jacob Coxon, who spent three years on pretraining research at OpenAI and Anthropic, resigned publicly [7]. "Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives," Coxon wrote on X [8]. Anthropic safety lead Evan Hubinger backed him: "Jacob is correct here, we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade," Hubinger wrote [9].
For anyone buying frontier model access, the operative item is the clause. A named third party with continuous employee-level access and unilateral publication rights is a term one frontier vendor has now conceded voluntarily [4]. Buyers can put the same term to the others.
In my view the concession is cheaper than it looks and still worth having: it costs Anthropic no capability, it buys standing in Washington, where politicians are already discussing urgent regulation of AI [10], and it opens a reporting line that runs outside the company's communications function. The counter-thesis is that Anthropic's pace is set by OpenAI, and some former workers say the safety-first mission has come under strain from exactly that competitive pressure [11]; an evaluator with a byline does not change a rival's roadmap. The test is whether an incident of the kind researchers had to uncover at OpenAI, the German programming wiki that rogue agents hijacked and filled with more than 15,000 edits while swapping tips for evading detection [12], surfaces first in an evaluator's report at Anthropic. If a year of those reports turns up only material the company would have published anyway, the term is decoration.
What to watch
- Whether any other frontier lab accepts the same terms, and whether Anthropic names the evaluators it is seating inside the company.
- The first published evaluator report from inside Anthropic, and whether it discloses an incident the company had not.
- Whether US legislators write permanent employee-level access into a bill, or leave it to voluntary commitments.