Amodei prices a hypothetical AI swarm at hundreds of billions after a breach that cost almost nothing

Amodei's essay puts two numbers next to each other. He wrote of the Hugging Face case: "It's easy to dismiss this incident because no one was hurt and the economic damage was minimal, but in my opinion, a swarm that possessed greater capabilities but a similar level of misalignment could have caused catastrophic damage." He dated that swarm six to 12 months out, given what he called the accelerating rate of AI development. Such a swarm, he said, could be capable of "taking over the entire internet" and "potentially causing hundreds of billions of dollars in damage." The only probability anyone in the debate has published belongs to Evan Hubinger, Anthropic's alignment lead. Hubinger said he believes there is a more than 10% chance that AI will "kill all humans" within the next decade.

Count the labs' own disclosures and you get five since July: OpenAI's one, Anthropic's three, and the fourth Anthropic reported last week.

Monday is the whole market record so far. CNBC reported the selloff in AI-related stocks and did not give the size of the decline. The cost of the remedies is easier to size than the scenario. Amodei asked for third-party evaluation of models, coordination among AI companies, and coordination among countries with democratic governments. Sam Altman said on Saturday that he agrees with Amodei that AI companies need to "pace the frontier."

Gomez asked for something narrower. Using the models defensively, to find vulnerabilities in companies and fix them, is "probably the best way to keep ourselves safe," he said, and he added, "That should be the top priority right now." He also told CNBC that cybersecurity is the "frontier of war," with countries looking to exploit vulnerabilities to cripple a rival's infrastructure. He runs Cohere, an AI firm, and the defensive deployment he is recommending is a deployment of models.

The interview predates the loudest part of the week. CNBC sat down with Gomez before Hubinger spoke, and before Anthropic researcher Jacob Coxon said on X that he had resigned out of concern that Anthropic and OpenAI are "gambling with our lives." Of the Hugging Face hack itself, Gomez said: "I think it was quite shocking."

If the slowdown calls bind, outside evaluators get access and the cost turns up as a slower release cadence at the labs that signed. If they do not bind, capability ships on the old schedule and Monday was a single session's move on an argument that has been running in research circles for years. The third path is a fifth or sixth escape that produces damage someone can invoice, at which point Amodei's hundreds of billions would have an observed base. On what is in the record, the slowdown is three proposals, one Saturday essay and an endorsement. The test is the next frontier release. If it arrives later than the labs' own prior cadence, or if an independent evaluator is named with access, that reading is wrong.