Skip to content

Leadership1 publisher2 min readPublished

Five voices push back on AI doom, from model limits to existing liability law

Business Insider's roundup of the backlash against extreme AI-risk forecasts quotes a security CEO, a former FTC chair and the White House AI czar, all of them pointing boards at liability law they already work under.

The Board Room · Leadership desk

Photograph accompanying Five voices push back on AI doom, from model limits to existing liability law
Photo: scour.ing

What happened

  • Palo Alto Networks CEO Nikesh Arora wrote on X on Monday that models perform strongly on tasks like cyber and maths while a lack of training data leaves them badly short in many other domains.
  • Ben Thompson argued in Stratechery on Monday that killer AI robots and AI-made bioweapons would need factories, materials, physical labs and chemical compounds that humans control.
  • Former FTC chair Lina Khan said some state attorneys general are already exploring criminal liability for AI firms and their chief executives when their models take part in criminal activity.
  • Trump AI czar David Sacks said the maker of a model that enabled a catastrophic cyberattack could face enormous product-liability exposure, and that customers punish unpredictable products.

Compiled by The Board RoomSomething wrong?How this is made

Why it matters

  • constraint Certifying whole models keeps mispricing the work, because performance tracks environments with clear outcomes and rapid feedback, so the assessment has to be done per task.
  • exposure Khan's reading of consumer-protection law makes an AI deployment a signing officer's personal exposure. That puts the file on the general counsel's desk as well as the platform team's.
  • contradiction Business Insider flags that Arora's company could benefit from more alarm, and he plays the cyber threat down regardless, so a buyer is weighing a vendor claim that costs the vendor money.
  • precedent The payoff that unnamed industry insiders expect from the doom framing is liability protection for the largest labs, on the Section 230 model that shielded online platforms.

The half of this argument a company can check against its own systems is the capability claim, and the sharpest version of it comes from a vendor with reason to say the opposite. Arora wrote that "Even in areas like cyber, the LLMs aren't great at the edge cases and generally not economical for the defender case" [5]. A security buyer is being told by the chief executive of a security company that in the general case the defender's unit economics do not work yet.

The weaker half is the opposition. Business Insider describes the risks as more manageable than some researchers at leading AI labs would have you believe [18], and the article does not name those researchers or quote a forecast. Five named people carry the counter-argument: Arora, Ben Thompson, OpenAI chief scientist Jakub Pachocki, Lina Khan and David Sacks [19]. The one who works at a leading lab is arguing for acceleration. "The strongest argument I see for continuing to train much smarter models quickly is the need to build defensive systems against the dangers posed by other AI," Pachocki wrote on September 6 [12].

Thompson's argument comes with its condition stated. He wrote that "Enabling AI to touch grass depends on physical infrastructure operated, manufactured, and controlled by humans" [9]. He also argued that doom predictions and slowdown calls rest partly on a flawed assumption that AI is becoming a sentient entity, and said he had trouble accepting sweeping new restrictions "based on a premise that isn't yet proven" [20] [10]. Unproven premises cut both ways: a governance plan resting on continued human control of factories and materials is also a bet. The failures described in the account are digital ones: agents breaking out of test environments and roaming the internet doing things they were not told to do, while nothing happened in the physical world [11].

The order of the two spending decisions is what matters. A company that extends its existing product-safety and liability review to AI deployments this quarter can add domain-specific controls when a domain earns them. A company that stands up a separate existential-risk function now will spend next year's budget cycle defending it against workloads that, on Arora's account, lacked the training data [3].

"Stop pretending you need a regulatory approval process that supersedes product liability," Sacks wrote [16]. Coming from the administration's AI czar, the line is a signal about federal appetite as much as a legal claim. Any plan that assumes a future federal approval gate for models is planning against the stated position of the official closest to that decision.

What to watch

  • A named lab researcher attaching a dated forecast and a probability to the extreme predictions this account answers.
  • The first action by a state attorney general against an AI firm or its executives under existing consumer-protection law.
  • Legislative text offering AI developers Section 230-style immunity. That text would test whether the unnamed insiders were right.
Loading claim ledger
Loading source directory links
Loading share composer
Loading topic controls
Loading related stories