OpenAI and Anthropic turn to small nonprofit evaluators whose funding is still unsettled

"To a degree, the problem, as always, is money," Suresh Venkatasubramanian, a computer science professor at Brown University, told CNBC. "Who is paying for these companies to do their work? How are they going to support them? You need an ecosystem, you need a viable business model for this." ^[5]

Most of the groups he is talking about are nonprofits ^[2]. So far their main job has been to assess what models can do and what risks they carry, and to flag cases where a model behaves badly ^[2]. CNBC reported that the groups are still getting established, even as capital pours into the industry at historic rates and new models are released at an unprecedented pace ^[3]. The report does not give a budget or headcount for METR, Apollo Research or Transluce.

So far the only named buyers are the labs being checked. OpenAI said on Friday it is "actively finalizing contracts with third-party safety assessors and will announce details in the coming weeks" ^[8]. A spokesperson said that work builds on existing collaboration with METR and Redwood Research ^[9]. Anthropic's chief executive started this round with his pledge to embed evaluators, and the company did not respond to CNBC ^[6][10]. The White House is offering encouragement. President Donald Trump praised AI executives for their "tremendous self-policing" and, in a voluntary accord presented in late September, urged them to "partner with an independent external auditor or evaluator" ^[7]. For now, CNBC reported, Anthropic, OpenAI and the infrastructure partners profiting from the boom are writing the rules ^[13].

The money has three obvious places to settle. Lab contracts could give a small evaluator defined access and a reporting line outside the company, so lab money pays for a check the lab cannot edit. The work could move to the larger firms already in the field, such as the accounting and auditing firm Accenture ^[11], and become a services line with an ordinary client relationship. Or a party other than the labs could pay for the ecosystem Venkatasubramanian describes ^[5].

We think the first is the hardest to reach. An evaluator sees only what lab staff are allowed to show it, so the lab's information rules limit the check whatever the contract says. OpenAI fired three employees last week for "violating our policies on accessing and handling sensitive company information," a spokesperson said. Two of them, Mikita Balesni and Tomek Korbak, said they believe they were dismissed over how they communicated with third-party evaluators ^[15]. "I worry the pervading fear to speak up and engage with third parties will mean OpenAI will cut corners on safety behind closed doors," Balesni wrote on X ^[16]. The counter-case is that OpenAI is fencing off sensitive data the way it would for any contractor, and that signed contracts will replace informal staff contact with something more durable. OpenAI disputed Balesni's characterization and wrote that it is "committed to embedding external assessors" ^[14].

We would be wrong if the contract details OpenAI has promised ^[8] give evaluators access and a reporting line that do not run through the company. Andrew Freedman, chief executive of the policy nonprofit Fathom, said the field is maturing quickly. "I've never seen an issue move so fast on so many different political spectrums," he told CNBC ^[17].