Anthropic and OpenAI pledge to embed outside safety evaluators with employee-like access

Amodei's proposal, as Fast Company describes it, is about access: independent evaluators sitting inside a frontier lab with employee-like reach, so the company is not the only judge of its own safety ^[2]. Anthropic's policy page says AI companies "shouldn't be the only ones deciding whether their systems are safe" ^[3]. PitchBook senior analyst Harrison Rolfes describes the result as permission. "It actually adds a very large cost because now you need a third-party stamp of approval to release models," Rolfes said ^[5].

Fees are one part of it. Evaluators also need computing resources, and the costs grow with the complexity of the model under test ^[13]. Marius Hobbhahn, CEO of Apollo Research, wrote in 2024 that "Since models are now capable enough of acting as LM agents, evals have to be increasingly complex tasks, which significantly increases the overhead per eval" ^[12]. Anthropic said much the same in a 2024 blog post, that the cost of evaluating huge and complex AI models "is high, and getting higher" ^[10]. No one quoted puts a dollar figure on a single frontier evaluation ^[22].

The body doing the regulating might not be a government. Fast Company reports the labs could instead agree among themselves on a set of independent evaluation firms, naming METR, Redwood Research and Apollo Research as candidates ^[8]. Rolfes compares that to the Big Four accounting firms, paid to vet other companies' financial statements ^[9]. A slate of approved evaluators agreed among the largest labs would be a standard the labs set themselves. Critics, in the same account, see the endpoint as an AI industry supplied by OpenAI, Anthropic, SpaceXAI, Google, Microsoft and perhaps Mistral in Europe ^[14], which is six companies, one of them outside the United States ^[20].

Wired's Maxwell Zeff reported on September 10 that OpenAI had asked members of Congress whether orchestrating an industry-wide slowdown in frontier AI development would violate antitrust law, citing people close to the company; that reporting does not say OpenAI asked about an agreement among industry players ^[19]. Two days later, Amodei's September 12 essay called for "pacing" frontier development so the labs could catch up on safety and alignment research ^[15][21]. Altman publicly agreed, saying "we need to pace the frontier" ^[16]. Musk's response was "Dario is right" ^[17]. OpenAI policy chief Chris Lehane later told reporters the company had already been working with Anthropic and Google DeepMind on AI safety for several weeks ^[18].

For a team deciding what to build on, whether you consume a model or publish one decides it. Call a hosted API and an evaluation regime arrives inside the vendor's price, invisible on your side. Ship weights that other people download and the stamp Rolfes describes lands on your release date, at whatever the evaluators charge ^[5]. For most roadmaps, what settles it is whether shipping requires an outside sign-off, and whether the schedule can absorb the wait for one.

What exists today is a proposal from one CEO and a public agreement from another ^[2][4]. No statute in this record requires a third-party evaluation, and the cost case rests on Rolfes plus Anthropic's own writing. Anthropic put that warning on the page in 2024: tests should be a "well scoped, small set", it said, "or else we'll create regulatory burdens and increase the possibility of regulatory capture" ^[11].