Invest1 publisher3 min readPublished
Washington rejects the global AI rulebook OpenAI and Anthropic asked the Security Council to write
Dario Amodei and Sam Altman told the UN Security Council that AI could threaten humanity and asked governments to set safeguards. The White House science adviser in the room said no new global governance structure is needed.
The Investor · Invest desk
What happened
- Anthropic's Dario Amodei and OpenAI's Sam Altman testified at the UN Security Council on Wednesday and said the world's governments have to set safeguards before AI becomes too powerful to rein in.
- Amodei asked leaders to rule out AI-built biological weapons, to work out how to verify what models can do, and to agree common standards for testing, loss of control and misuse.
- White House science adviser Michael Kratsios told the Council that worries about control are no reason to pause AI development or to constrain it with a new global governance structure.
- China's UN ambassador, Fu Cong, said every country should be free to choose its AI technology and urged governments to promote open-source models, which he described as China's approach.
- Ed Miliband said Britain will put controlling AI at the heart of its G20 presidency next year and will look for a single set of global standards for safe development.
Compiled by The InvestorSomething wrong?How this is made
Why it matters
- contradiction The labs asked the Council for binding standards and the American government in the same room refused the structure, so anyone pricing regulatory risk into these firms is pricing Washington's position.
- exposure Two frontier chief executives are now on the UN record describing their own product as a possible risk to humanity, words available to any legislator or plaintiff later arguing the firms understood what they were shipping.
- decision With the United States rejecting a new global structure, the venue that can actually enforce anything is national, and the labs' government-affairs budgets follow the capitals.
Dario Amodei said this about the product his own company sells. "If managed poorly, I even believe AI could be a risk to humanity as a whole," the Anthropic chief executive told the Council [2]. Sam Altman put it this way: "We could lose control of the future to AI" [3]. Both men also asked the world to keep the power of AI from being concentrated in one company or country [5], and Altman described the upside as "giving people more power over their own lives" [19].
A verification requirement has a price. Proving what a model is capable of doing, to a common standard, is a fixed cost per model [6], and a fixed cost per model is cheapest for the firm with the most revenue behind each release. Hugging Face's Clement Delangue, the third executive on the agenda, appeared over a glitchy remote connection [13]. China's ambassador, Fu Cong, warned the Council against "banding together into a petty us-vs.-them clique in the tech sector and forcing other countries to take sides" [10].
France convened Wednesday's meeting [14]. Its foreign minister, Jean-Noel Barrot, said: "Just as with the atom back then, the international community must now regulate AI in order to make the most of its potential and avoid the worst" [15]. Count the governments the AP account quotes and the split is four to two, with Britain, France, Liberia and Ukraine for controls and the United States and China declining a new global structure [20]. Michael Kratsios, the White House science adviser, offered the room something else: "We want you, our allies and partners, to share in the benefits of this technology" [8].
The only casualty figure in the record is a battlefield one. Kentaro Toyama, an information technology professor at the University of Michigan, wrote in an email that a Russian drone last month "autonomously made a targeting decision that killed three Ukrainians. Killer robots are technically easy to build, and they are real. We should not be under any illusions otherwise" [17]. Antonio Guterres, the UN secretary-general, had warned of "killer robots" the day before [18]. Volodymyr Zelenskyy, whose country has been at war for four and a half years [22], told the General Assembly: "There is already a real possibility that AI, not only people, will begin to decide what happens on the battlefield. And we need peace before we reach that point" [16].
By the AP's account, the session produced no commitment, instrument or date. Ed Miliband, the British foreign secretary, said the executives "are telling us in stark terms that it cannot be left to them to prevent disaster from befalling the world," and added: "We cannot ignore them because they are right" [11]. An investor marking down Anthropic or OpenAI for regulatory risk on the strength of Wednesday is marking down a speech. A common testing standard is also a barrier to every lab that cannot afford to pass it, and Anthropic and OpenAI, the two firms best placed to pass one, asked for it [6].
What to watch
- Whether Britain's G20 year produces a draft testing standard with a named enforcement authority and a compliance date.
- Whether the American position changes, since Kratsios speaks for the jurisdiction that hosts both firms.
- Whether any government converts the bioweapons red line into a pre-release verification requirement labs must pass.