Product1 publisher3 min readPublished
Anthropic publishes the counting methods behind its slowdown proposal
Five days after Dario Amodei asked rival labs to coordinate on slowing the frontier, Anthropic detailed three measurements it already takes, including a count of about 30,000 autonomous agents in its own environments.
The Product Desk · Product desk
What happened
- Anthropic said it measures AI-led R&D, oversight of autonomous agents and compute allocation, and it published the exact methodologies it uses so other frontier labs can take the same readings.
- Anthropic put the number of autonomous agents running in its computing environments at around 30,000, with most of them involved in its R&D work.
- An internal index of how involved Claude is in Anthropic's R&D processes found the model is not operating fully autonomously in any subset of that work.
Compiled by The Product DeskSomething wrong?How this is made
Why it matters
- precedent By publishing method and not only results, Anthropic sets the floor for what a rival lab's progress report has to contain before anyone can put the two side by side.
- exposure Once the counting method is public, a lab that never reports an agent count is declining a measurement its competitor has shown is possible.
- constraint The autonomy claim rests on an index Anthropic designed and can revise, so any comparison over time holds only as long as the definition does.
- decision Enterprise buyers get a new line for the vendor questionnaire: which of these three numbers you produce, from what instrumentation, and how often the snapshot is taken.
Ask a team how many agents are running inside its own environment and you tend to get three numbers from three dashboards. None of them comes from a system that could stop an agent mid-action.
Anthropic said it built that system: software able to oversee AI agents and intervene on the actions they take within its internal systems [5]. For anyone copying the approach, the sequence matters more than the total. The intervention point comes first. Once every agent action runs through one place, you can count them.
The R&D measurement answers a narrower fear. Anthropic said it tracks AI-led R&D because the best labs use AI to build more powerful models that build on their own capabilities. The risk is that humans fail to understand how dangerous those models are [15]. Evan Hubinger, one of the company's own safety researchers, warned this month that he estimates a greater than 10% chance AI "could kill all humans" within the next 10 years. He stressed that the risk from current models is still very low [12]. Hubinger said he worried about the technology's potential to develop and improve itself to the point where humans can no longer control it [13].
The compute measurement is the one designed for people outside the building. An autonomy index is an instrument the lab designed and runs; chip allocation leaves records in other people's ledgers. Anthropic said it has started taking regular snapshots of how compute is allocated [9]. It wrote that "compute is among the most verifiable inputs to the AI R&D process, meaning that it could be a critical lever in a future pacing effort" [8].
Amodei outlined his three-step plan on Saturday 12 September, and the methodologies went up on 17 September, five days later [2][3][1]. SiliconANGLE reports that Sam Altman, Elon Musk and Demis Hassabis are among the peers backing that plan, and every measurement in its account is Anthropic's own [4][3]. Amodei said he wants to curtail the pace of frontier model development "without sacrificing commercial advantage of the United States' lead in AI" [14].
Anthropic put the case for publishing plainly. "As the world considers pacing the frontier, we should do everything possible to minimize the gap between what frontier labs know and what the public knows," the company said. It added that this means "better measuring the development of AI, reporting on it publicly and giving society an opportunity to decide how to use this information" [10][11].
For teams that will never run a frontier lab, the transferable part is small and specific. One of the three metrics needs new plumbing before it can produce any number at all. The agent count depends on a control point, and that costs engineering time and adds latency to every agent action [5]. Compute allocation is bookkeeping you already have [9]. When a vendor starts quoting figures like these on a trust page, ask which of them came out of a system that can stop an action, and which came out of asking the engineers.
What to watch
- Whether OpenAI or Google DeepMind publish an agent count and an autonomy index using Anthropic's stated methods.
- Whether the compute snapshots turn into a scheduled public disclosure with a fixed cadence and an outside reviewer.
- Whether Anthropic reports how often its oversight system actually intervenes on an agent action.