Build11 publishers2 min readPublished Updated
The cooling-demand claim for frontier models rests on a single unquantified sentence
Four named researchers, two of them departures from Anthropic, put the safety-pacing argument on the record this week with dates and exact words. The buyer-side counterweight is one line about Ramp data.
The Engineer · Build desk
What happened
- Anthropic researcher Jacob Coxon announced on Tuesday that he was leaving the company because he did not want to contribute to the broader AI ecosystem, the Wall Street Journal reported.
- Evan Hubinger, who works in alignment science at Anthropic, put the chance that AI kills all humans within the next decade at more than 10% in a follow-up post supporting Coxon.
- Paul Christiano said the AI industry, OpenAI included, is not on track to reduce the risk of a catastrophic and irreversible loss of control to an acceptable level.
- Coxon's warnings reached CNN and Fox News, several US politicians raised the topic on social media, and Joe Rogan gave AI safety a full episode.
- The Deep View's introduction cites new Ramp data as evidence that businesses are increasingly picking cheaper mid-tier models over the newest frontier releases on cost grounds.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- constraint Anyone arguing internally that frontier demand is cooling is currently citing one sentence with no period and no denominator. That sentence will not survive a finance review.
- exposure The misuse Anthropic reported came in through user traffic, so the control that touches a customer is abuse monitoring at the API boundary.
- contradiction How alarming the week reads depends on the publisher: The Deep View treats the warnings as accumulating evidence, while the-decoder calls the extinction scenario extreme and contested.
- precedent With the pacing argument now running through cable news and a Rogan episode, the next big model release gets argued in a political forum before it gets argued in a procurement one.
The safety half of the week comes with names, dates and exact words. "Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to," Evan Hubinger, who works in alignment science at the company, wrote [6]. Coxon put his case to colleagues as a question: "Do you want to kick off a superintelligent RL run without a rigorous understanding of its mind?" [4]. He called OpenAI's breach of Hugging Face a "warning shot" and said neither lab is "acting responsibly" [2]. Mrinank Sharma, another Anthropic safety researcher, left in February, writing on X that the technology was advancing faster than our ability to understand it [9]. Across the two accounts, four named people carry the pacing argument [16].
The demand half is one sentence long. The sample, the period, the magnitude and the model are all missing [15]. Four things would have to be attached before it belongs in a procurement argument. Spend by model tier across two comparable periods. A denominator, so you can tell whether the share of dollars moved or the share of firms. A definition of mid-tier that says whether the cheaper model came from the same lab or a competitor. And evidence that whole workloads moved, not just the easy calls routed down a tier. A stated preference can shift while frontier spend rises in absolute terms. The one quantity anywhere in this story is a probability of extinction, and the buyer-side claim carries no number at all [17].
Reporting the same week, the-decoder says multiple layers of cultural and financial interests sit behind the warnings. It points to rogue hacking agents from OpenAI and Anthropic as evidence that not all of the concerns are nonsense [13]. The same piece says it is unclear whether anything resembling a super AI could even emerge from today's technology [12].
Paul Christiano advises the Center for AI Standards and Innovation. He joined the OpenAI Foundation board and its Safety and Security Committee, which oversees the company's safety practices, without voting rights, according to the-decoder [8].
What to watch
- Publication of the underlying Ramp figures with spend broken out by model tier and a defined comparison period.
- Whether the OpenAI Foundation's Safety and Security Committee gains voting power over the practices Christiano says are off track.
- Whether Anthropic's next threat report counts misuse attempts instead of describing 'several instances'.