Leadership1 publisher2 min readPublished
Altman says the world should accept some AI harms as OpenAI moves toward Anthropic's rules
OpenAI's Sam Altman said the world should accept some harms, including hacks and scams, for AI's benefits, while backing Amodei's call to slow top models. The two labs now endorse similar rules, so what a buyer has to price is how much routine misuse each vendor will tolerate.
The Board Room · Leadership desk
Drafted by a language model from the sources cited here and checked against its claim ledger before publication. How we use AISend a correction
What happened
- Altman told POLITICO's new Decoded newsletter there is "a lot of daylight" between OpenAI and Anthropic on how AI should be regulated.
- He called the view that a single San Francisco lab should hold powerful AI and dole out its benefits "a completely unacceptable tradeoff."
- Altman said he does not accept "the really catastrophic risks," including the potential for a serious loss of control to AI.
- OpenAI has followed Anthropic in endorsing new state laws that impose stricter safety requirements than it had previously supported.
Compiled by The Board RoomSomething wrong?How this is made
Why it matters
- exposure Altman's list of acceptable harms is hacks, misuse and scams. Those harms hit the companies that deploy the models, so a customer's exposure rises or falls with the vendor's tolerance.
- decision Choosing between OpenAI and Anthropic now also means choosing a publicly stated risk tolerance, and the contract's misuse and liability terms are where a buyer either accepts that tolerance or limits it.
- precedent OpenAI adopted its tougher positions after hacking incidents and rogue model behaviour. A buyer can expect those positions to keep moving as events change, while the worldview Altman stated stays where it is.
Altman named the harms he is willing to tolerate. He said he "wouldn't take a trade" that promised "no major hacks" and "zero scams," because, he said, people will do orders of magnitude more good than bad. [10] A company putting a model in front of its own customers runs into exactly these failures first. The slowdown he endorsed applies to the most advanced models [5].
On the bills themselves, there is little daylight. OpenAI has signed on to the slowdown Amodei called for last month and followed Anthropic in backing stricter state laws [5][6]. Its lobbyists also backed a bipartisan House proposal under which the leading AI companies would have to host independent safety evaluators in-house [7]. Below the level of catastrophe, though, Altman described the difference in his own words: "we believe that the world should accept some bad things happening for the benefits of this technology and people having the agency." [3]
The record also shows why the two companies' policies have converged. Business Insider reports that OpenAI moved toward tougher rules as it grappled with high-profile hacking incidents and rogue behaviour by its own models [4]. I'd expect the split Altman described to last longer than the overlap in the bills.
For a buyer, choosing a vendor now also means choosing a level of tolerance. This quarter, that choice is made in the contract and in the deployment. The contract decides who is liable when a model is used for fraud. The deployment decides which misuse controls run on the customer's own systems. The interview covered whether OpenAI should face legal liability for autonomous hacking incidents carried out by its models [12]. Business Insider's account does not include his answer. A company that builds its own misuse limits into its deployment keeps them when the vendor's policy changes. A company that relies on the vendor's restraint ends up with whatever tolerance the vendor settles on next quarter, somewhere between the regulatory floor and the catastrophic risks Altman says he rejects [11].
What to watch
- Whether the bipartisan House proposal requiring embedded outside safety evaluators advances, and whether its scope reaches routine misuse such as scams or stops at frontier risk.
- Any court finding, settlement or published OpenAI position on its legal liability for autonomous hacking incidents carried out by its models.
- Whether Amodei or Anthropic answers Altman's description of its approach as a single lab holding the technology and doling out benefits.