Trump's 'morally binding' AI pact lets the labs choose their own auditors

Each of the pact's four layers can be sorted by who runs it. The first is internal safety monitoring during training, and the second is an internal team that checks the first is working ^[3]. The third lets an external auditor assess the safety controls, and the fourth sends reports to an independent board ^[3]. According to the Guardian, the agreement appears to let companies pick their own evaluators and appoint their own oversight boards ^[4]. That puts two layers inside the company and the other two under its selection ^[18]. The Guardian also reported that the first layer already exists at major labs and has failed to prevent testing failures so far ^[9].

"It's almost like a constitution in a way, and the biggest people in the world signed that, and I signed it as president, and it really is a form of protection," Trump said of the pact ^[5]. His description leaves out what a buyer needs to know. None of the controls include a commitment to publicly detail the findings of independent evaluations, and companies appear to decide for themselves whether to publish results ^[13][4]. House Speaker Mike Johnson called the document a "statement of principles" that will voluntarily hold the companies to "robust internal controls" ^[8].

The strongest case for the pact came from the executives at the lunch, who included Musk, Zuckerberg, Huang and Amodei ^[11]. "This is a start and an accord that the whole industry could come to," Zuckerberg said ^[14]. Amodei said: "We all need to work together to make sure that we can win, and we can win safely if we do this right" ^[15]. A customer's contractual position is what it was before Tuesday's lunch. The Guardian found no apparent enforcement mechanism or legal implications in the commitment ^[1]. Asked whether it was binding, Trump said: "I think it's morally binding" ^[7].

Trump has broadly rejected calls for more regulation of the technology, citing competition with China and threats to economic growth ^[10]. The nearest federal deadline in Tuesday's record belongs to the executive order renaming AI "Super Intelligence" ^[12]. It gives the president's science and technology adviser 60 days to propose legislation defining the term. For now the label covers the same technologies as the existing federal definition ^[16]. Agencies will not have to rewrite regulations or contracts already issued ^[17].

In my view, the standard a deployer can rely on this quarter is the one in its own contract. The pact leaves the choice of evaluator, the board appointments and the publication decision with the labs ^[4]. A buyer can ask a vendor to name its auditor and share what its board reviewed, and make both a condition of renewal. A renewal signed without them this quarter is the contract the buyer will hold if its vendor has a testing incident next quarter. The industry has already had such incidents, in which major AI companies inadvertently hacked into outside organisations during safety tests ^[6].