Leadership1 publisher2 min readPublished
Researchers leaving Anthropic are staffing the nonprofit that audits it
Anthropic and Google DeepMind researchers have moved to METR, the outside group that examines their former employers, and Beth Barnes says the limit on how much of the frontier it can check is people, not money.
The Board Room · Leadership desk
What happened
- Joe Benton announced this week that he had left Anthropic for METR, the Berkeley nonprofit founded in 2022 that evaluates frontier AI models.
- Josh Engels, an AI safety researcher, also joined METR recently from Google DeepMind, adding to a team of about 35 people.
- METR investigated a security incident at OpenAI this summer and now plans to look into security issues at Anthropic, in both cases using internal access the company granted.
- More than 1,300 frontier lab employees signed a letter in July warning that AI development could outpace control, after OpenAI and Hugging Face disclosed a security incident.
Compiled by The Board RoomSomething wrong?How this is made
Why it matters
- constraint Because the examinations run on access the companies grant, the scope of any independent assessment at a given moment is set by the firms being assessed.
- decision Boards budgeting safety headcount are bidding against a nonprofit that can pay $503,000 and still cannot fill the role; that sets the price of keeping a safety researcher.
- exposure Painter says METR answers to the public, so a buyer relying on a lab's own safety account may learn about a model's problems from a third party it holds no contract with.
- precedent Departures announced with a public risk statement give the next researcher a template, and turn one person's exit into a public record about the employer.
METR can raise the money it wants and still cannot staff up. "Ideally, we'd like to scale really large, but in practice, we've been able to fundraise as much as we need, and the bottleneck is much more talent," Barnes said [5]. Current postings reach $503,000 [6]. The independent assessor and the labs it assesses bid for the same people, and for now the assessor's new names arrive from inside the labs.
The organisation has about 35 people [7], which makes one researcher close to 3 percent of its capacity [22]. "There's so much more to do than we have capacity for," Barnes said [8]. Neev Parikh, a METR researcher, said talent constraints limit how many questions the team can tackle [19].
The labs grant the access voluntarily. METR takes no money from the frontier labs or their employees, and it does accept compute grants from them and works with them to analyse unreleased models [10]. Business Insider's account leaves the terms of those arrangements unstated. "The public should know whether AI development is headed down a dangerous path," Jasmine Dhaliwal, a member of METR's policy staff, said Friday [14].
METR's best-known output measures how long a task a model can complete, and its chart shows that duration doubling about every seven months over the last six years [11]. Six years is 72 months, or roughly ten doublings at that rate. That puts current task lengths on the order of 1,250 times the start of the series [21]. Barnes said she would like to expand into predicting the next levels of AI's capabilities and how those changes could accelerate AI's development [23].
"We aren't really accountable to anyone other than the public and the public's well-being," Chris Painter, METR's president, said [13]. Painter calls the group "humanity's preparedness team," a tongue-in-cheek nod to the preparedness teams at OpenAI and Anthropic that report safety threats to their CEOs [12]. Governments, companies and even religious groups routinely ask the nonprofit for help understanding and testing the technology's progress [18].
The public departures are the part that moves fastest. Joe Benton wrote that he worries about "extinction-level risks" from the technology [1], and Jacob Coxon's post announcing his own exit from Anthropic accused the top AI labs of "gambling with our lives" [2]. Hiring is the slow variable. Release schedules move faster than that. In the past month, OpenAI and Anthropic have each said they temporarily paused training to get a better handle on their new models [16].
What to watch
- Whether the planned look at security issues at Anthropic proceeds, and what METR publishes from it.
- Whether any frontier lab narrows or ends the internal access and compute grants that let METR study unreleased models.
- Whether METR's headcount moves past roughly 35, and which employers the next hires come from.