Security1 publisher2 min readPublished
Huang assigns the frontier release decision to the AI companies themselves
The Nvidia chief said at a Salesforce conference in San Francisco on Tuesday that safety is an engineering problem, while Amodei, Altman, Hassabis and Musk want governments involved and OpenAI drafts standards it calls voluntary.
The Watch · Security desk
What happened
- Jensen Huang said AI does not need new laws or regulations, and that there should be no false choice between the speed of innovation and the safety of AI products.
- Anthropic's Dario Amodei called over the weekend for the pace of all AI development to slow and urged governments to regulate the industry.
- Amodei said Anthropic is in a dialogue with the rest of the industry about committing to better safety standards and checks, work the leaders of OpenAI and Anthropic describe as recently started.
- Patrick Hillman of Logical Intelligence said companies that believe what they are building is dangerous should show what they are prepared to stop doing.
- Other AI executives and experts said they agreed with Huang, and some industry figures said the extinction fears were overblown and were being used to generate hype.
Compiled by The WatchSomething wrong?How this is made
Why it matters
- constraint When a model version ships on the supplier's own confidence, the assurance file a deploying team keeps for that change contains the vendor's judgement and no external test result.
- decision Buyers who want release criteria have to negotiate them into contracts now: notice before a model version changes, and the right to stay on the version they tested.
- contradiction Huang treats safety as engineering while Amodei asks legislatures to act, so a supplier safety attestation means something different depending on which camp wrote it, and a questionnaire that scores them the same measures nothing.
- precedent If the labs publish their voluntary text first, that text becomes the reference an auditor asks about and the baseline any later rule gets written against, with the labs choosing the criteria.
Nvidia is the biggest company in the world by valuation [16], and its chief executive spent Tuesday arguing that the heads of AI firms are the right people to decide whether a new version of the technology gets released [2]. "Safety is paramount. However, safety is an engineering problem," Huang said [4]. "Run as fast as you can, but if at any time you feel the institution is not in control, take a pause," he said [5]. A leader who lacks confidence in a product should hold it back, he said, and called that "a very obvious thing to do" [6].
Under that test the release decision stays inside the company and moves on the company's schedule. By name count the other camp is larger, four executives to Huang's one [18]: Amodei urged governments to regulate the industry [7], and Altman, Hassabis and Musk applauded him [8]. The BBC report does not identify a bill, a jurisdiction or a deadline [19].
The process that exists is private. OpenAI is working with other labs, among them Anthropic and Google DeepMind, "to advance frontier AI standards, building a voluntary effort now, with or without government support", the company's Chris Lehane said last week [10]. "With stakes this high, we cannot let the perfect become the enemy of the good," Lehane wrote [11].
Patrick Hillman, chief business officer of Logical Intelligence, which is chaired by Yann LeCun, said self-regulation asks for trust the industry has not earned. "The only institution that Americans might trust less than Washington these days is Silicon Valley. I have worked and lived in both and I assure you both have earned this scepticism," Hillman said [12].
Amodei, appearing at the same conference, said the bigger surprise of the boom was not the technology's advance but its reach. "We didn't appreciate that it would lead to companies growing so fast, and how quickly they would become central to things," Amodei said [17].
What to watch
- Whether the OpenAI, Anthropic and Google DeepMind standards effort publishes text with testable release criteria, and whether any of it reaches customer contracts.
- Whether any government attaches a statutory hook to the voluntary frontier standards Lehane described.
- Whether a lab actually exercises the pause Huang described, and discloses that it did.