Product1 publisher3 min readPublished
Anthropic and Nvidia split over AI rules on Salesforce's Dreamforce stage
Benioff showed a graph projecting more than $46 billion of 2027 revenue helped by AI demand. Minutes later Anthropic's Dario Amodei asked for a paced frontier and Nvidia's Jensen Huang said no new laws are needed.
The Product Desk · Product desk
What happened
- Marc Benioff opened Dreamforce by calling up a revenue graph showing Salesforce projects it will top $46 billion in 2027, credited in part to demand for its AI-powered products.
- Moments after Amodei left, Nvidia CEO Jensen Huang told the same audience that AI safety amounts to an engineering problem and that AI companies can police themselves.
- The argument reached Dreamforce a week after AI researcher Jacob Coxon resigned from Anthropic and said publicly that companies racing toward self-improving AI were gambling with our lives.
- Amodei has urged world leaders to help pace the frontier of AI development, and other AI chief executives including OpenAI's Sam Altman began endorsing the idea.
Compiled by The Product DeskSomething wrong?How this is made
Why it matters
- decision A team signing Salesforce agent workflows has to decide whether it can live with a slower improvement curve on the model underneath, since the supplier of that model is asking governments to set the pace.
- constraint Huang's standard leaves the pause entirely inside the vendor's judgment. The customer has no contractual right to advance warning, so the continuity plan has to work without it.
- exposure Treating agent scope breaches as security incidents puts them on the customer's incident response rotation.
- contradiction Two suppliers spoke minutes apart on the same stage and disagreed on whether any external rules are needed. Buyers are left to reconcile a roadmap risk their vendors have not.
The person who has to make this work signs a Salesforce contract. The capability inside it comes from somebody else's model. By Wired's account, Anthropic is a critical partner in Salesforce's move into AI, and Amodei used part of his time onstage to plug Claude's integration with the product [3][5].
Huang's pause is the vendor's own call. "You run as fast as you can, but if you feel that at any given point in time, the company is out of control or the products are not going to be safe, you take a pause," he told Dreamforce attendees [8]. He also said AI safety amounts to an "engineering problem" and that the companies building AI can police themselves [6]. That standard keeps both the judgment and the timing on the vendor's side: the customer gets no role in it and no notice period.
Amodei asked for the pace to be set outside any one company. He said the responsible move after a rival's safety incident is to "organize the rest of the industry and say, what can we do to set standards for everyone?" [4]. He was careful to add that pacing the frontier does not mean "freezing the technology in place" [5]. That distinction is the part a buyer can act on: a paced frontier means a slower improvement curve on a model you already depend on.
The politics around it are unsettled. White House AI adviser David Sacks called Amodei's suggestion "just another bid for regulatory capture" [11], President Trump called existential AI risk fears "a hoax" [12], and Congress is pushing ahead with legislation to regulate the industry anyway [13].
The version of this fight that reaches an operations team is the one Sayash Kapoor and Arvind Narayanan describe. Their essay this week takes on "loss-of-control incidents," including OpenAI accidentally letting its agents hack into Hugging Face [14]. "Part of what we were trying to do in this essay is to move past this false dichotomy to bridge the ground between AI safety and cybersecurity," Kapoor said [15]. If an agent going outside its scope is a security event, it lands on the team that already runs incident response, with a rotation and a postmortem.
The slowdown argument reads like public policy, and it reaches the workflow that ships on Monday. Two things are worth marking on every agent workflow now in flight. First, whether it depends on one model supplier or can be swapped. Second, whether a frozen or paused model degrades something cosmetic or something you owe a customer under contract. The workflows sitting in the single-supplier, contractual corner are the ones I would raise with an account team now, and the ask is plain: what happens to this workflow if the model behind it stops improving or stops being available. The tradeoff for fixing it yourself is real. Making a workflow model-agnostic means re-running your evaluations and losing tuned prompt behavior. That re-evaluation work is the price of keeping the pace decision yours.
Wired reported the executives and the argument, and names no customer who changed a purchase because of it [17].
What to watch
- Whether the AI legislation Congress is pushing sets any pacing obligation on model suppliers, over objections from Sacks and the President.
- Whether Salesforce adds model-substitution or continuity language to agent contracts after its main model partner argued publicly for pacing.
- Whether enterprise security teams start logging agent scope breaches as incidents under the loss-of-control category Kapoor and Narayanan describe.