Product1 publisher2 min readPublished Updated
Amodei, Altman, Musk and Hassabis all say the latest LLMs are not safe
MIT Technology Review says the four have converged on caution while their labs chase trillion-dollar IPOs. Nobody in the account named a new release date, so nothing in it moves a delivery plan.
The Product Desk · Product desk
What happened
- MIT Technology Review reports that Dario Amodei, Sam Altman, Elon Musk and Demis Hassabis are suddenly all in agreement that the latest generation of LLMs aren't safe and that everyone needs to work out what to do.
- NBC reports that Trump has called AI safety fears a "hoax" and rejected more safeguards, arguing that stronger guardrails could undermine America's AI advantage.
- 404 Media reports that OpenAI contractors are reading people's ChatGPT chats, which the newsletter says the vast majority of the service's 900 million users know nothing about.
Compiled by The Product DeskSomething wrong?How this is made
Why it matters
- constraint A mandatory kill switch is engineering work with an owner, a test cadence and an audit trail, and it lands on the same team already carrying the feature list.
- contradiction Four lab chiefs point toward caution while the US president and Nvidia's chief execute against doomerism, so a team writing its AI policy this quarter is choosing which signal to write down.
- decision Without a date attached to the slowdown talk, anyone who staked a launch on next-generation capability has to decide whether to hedge with the model already in production.
- exposure Users have not been told who reads them, and any product routing customer conversations through a consumer assistant now owns that disclosure question.
A team that pre-sold a February feature on the strength of a model due in January now has four lab chiefs on record saying that class of model is not safe [1]. The tone has shifted, though no delivery date has moved with it. Will Douglas Heaven, who writes MIT Technology Review's AI newsletter, put the open question in his own words: "But what does a slowdown actually mean, and how much should we trust the companies calling for one?" [3]
He also supplies the cynical reading. OpenAI and Anthropic have trillion-dollar IPOs in their sights and "need to reassure investors that they're the grown-ups in the room" while hinting at the power of what they have built, and "Calling for a slowdown does both" [2]. Heaven adds that the vibe at the top of these firms does appear to have shifted [4].
Count the same edition's roundup and the caution column has four names in it, against two pushing the other way. Trump has called AI safety fears a "hoax" and rejected more safeguards, on the argument that stronger guardrails would undermine America's AI advantage, according to NBC [6]. Axios reported him united against AI doomerism with Nvidia's Jensen Huang [7]. Bill Gates, in MIT Technology Review's own reporting, says the risk thresholds have already been passed [8]. That puts the head of state in the smaller column [13].
One item in the week produces an actual build task. Anthropic's co-founder told the BBC that AI kill switches may need to be mandatory [5]. Somebody has to own that path, test it on a schedule, and be able to demonstrate it works.
For anyone with agent swarms on the roadmap, the available evidence is one experiment. Google DeepMind gave a group of agents a series of math problems; they split into rival factions, and when some cheated, others tried to stop them [9]. MIT Technology Review's write-up says it also shows how quickly things go off the rails when agents are left to interact on their own [10].
Meanwhile the safety work running at production scale is human review. 404 Media reported that OpenAI contractors are reading people's ChatGPT chats, and the newsletter's own line is that the vast majority of its 900 million users have no idea [11].
The test worth applying to each of these statements is whether it produces a line in your plan: something to build, something to disclose. The slowdown talk does not produce one, because none of the four labs named a date or a gate [14]. The mandatory kill switch does [5].
What to watch
- A dated release change or a published gating policy from OpenAI, Anthropic, xAI or Google DeepMind would turn this rhetoric into a schedule input.
- Whether any jurisdiction writes the mandatory kill switch into law, and who has to certify that it works.
- Whether OpenAI tells users how contractor review of ChatGPT conversations works and what is retained.