Product1 publisher3 min readPublished
Every AI safety control on this week's record was written by a company that sells models
Trump says the only guardrail AI needs is a strong and smart president. The controls a product team can actually cite are Anthropic's own plug-pulling, Microsoft AI's new outline and Monday's market selloff.
The Product Desk · Product desk
What happened
- Microsoft AI published an outline on "humanist AI", and Mustafa Suleyman told CNBC the firm had worked on it for months and published now because of the current debate.
- Warnings from executives and staff at leading AI firms triggered a selloff in shares of some tech firms on Monday as investors weighed the impact of a potential slowdown.
- Dario Amodei called for slower, more closely monitored development while saying any action to rein in AI should be done "without sacrificing commercial advantage".
Compiled by The Product DeskSomething wrong?How this is made
Why it matters
- constraint With no federal rule to point at, the answer a team gives on stopping a model rests on a vendor document the vendor can revise, and the team carries the difference in front of its own customers.
- decision Every team building on a frontier model now picks between owning a stop path in its own product or accepting that the only stop path sits inside a lab's internal process.
- exposure A roadmap that assumes faster models each quarter is exposed to the labs' own braking and to investors repricing it, not to a regulator's timetable.
- contradiction Anthropic's chief executive asks for a slowdown that keeps commercial advantage intact while the White House calls the concern a hoax, leaving a buyer reading both with no pace to plan against.
Somewhere a customer security questionnaire has a row asking who can stop the model if it starts behaving badly, and whoever fills it in has to cite something. As of Monday, everything available to cite was written by a company that sells models.
Anthropic co-founder Jack Clark told the BBC that a "kill switch" for dangerous AI that can be checked by a third party may need to be mandatory for the industry [4]. He said "most labs have different ways of being able to pull the plug", including Anthropic, but that lawmakers may need to enforce having one [5]. Shutting AI software off completely, he said, was something society "might want to eventually pass rules around" [6].
Washington is going the other way on that enforcement. Trump said fears about the safety of AI are a "hoax" and criticised calls for more guardrails [1]. On Monday he posted that the only "guardrails" AI needs was a "strong and smart" president [2]. He also wrote: "There is a SICK conspiracy going on against AI and Data Centers, and the only one that is happy about it is China. WHOEVER WINS AI, WINS!" [3]
Anthropic has internal ways of pulling the plug [5]. Clark proposes making a third-party-checkable version mandatory [4]. Microsoft AI published an outline document on what it called "humanist AI" [8]. Dario Amodei called for slower, more closely monitored development, and Sam Altman of OpenAI and Elon Musk of xAI said they agree with him [10][12]. Four items. All four were authored by companies that sell models, and none is a rule in force [18].
Mustafa Suleyman, chief executive of Microsoft AI, told CNBC that the firm had been working on the new guidance for months and had decided to publish it now due to the current debate around the technology [9]. Investors moved on the same debate: warnings from executives and staff at leading AI firms triggered a selloff in shares of some tech firms on Monday as investors weighed the impact of a potential slowdown [7].
The slowdown on the table has a condition attached to it. Amodei said any action to rein in AI should be done "without sacrificing commercial advantage" [11]. Xin Qiang is a professor of American Studies at Fudan University. He told the Global Times: "They know that China has become a strong competitor in AI, and worry that if the US slows down development or tightens regulation, China could catch up even faster." [15]
The BBC report does not mention enterprise procurement terms or any non-US regulator [19]. It carries a market reaction, four vendor documents and one senator, Bernie Sanders, who told BBC Newsnight last week that "we have got to do something immediately to stop the uncontrolled growth of AI" [17].
For the person shipping the feature, the sort that helps is two columns: who wrote the control, and who can change it without telling you. Right now almost everything goes in the same box. A vendor's stop procedure is not a control you operate. The feature flag in your own product is, and so is a second model you have already tested a fallback against. Those are the parts to work on this quarter. Clark's third-party-checkable version, the one an auditor could look at, is still a proposal [4].
What to watch
- Whether Microsoft AI's "humanist AI" outline turns up in its customer contract terms or stays a published document.
- Whether any lawmaker introduces Clark's third-party-checkable kill switch as an actual requirement.
- Whether Tuesday's Pro-Human Assembly, with Bannon and Sanders on the bill, produces a specific legislative ask.