Invest1 publisher3 min readPublished
Coxon, who called AI labs irresponsible, asks Congress to let leading labs regulate themselves
The former Anthropic researcher whose resignation post pushed a bipartisan group of lawmakers toward legislation spent Sunday arguing that the frontier labs should be left to slow themselves down while Congress writes a framework.
The Investor · Invest desk
What happened
- Jacob Coxon, the former Anthropic pretraining researcher, told NBC News on Sunday that Congress should allow the leading frontier labs to regulate themselves while lawmakers write a long-term framework for slowing AI development.
- He said the bills already introduced in Congress, including the framework for a national kill switch triggered when AI exceeds the bounds of human control, will quickly be outpaced.
- Axios reported on Friday that House members were circulating a letter asking Speaker Mike Johnson to bring the chamber back into session earlier than planned to pass a set of AI safeguards.
Compiled by The InvestorSomething wrong?How this is made
Why it matters
- contradiction The same witness now underwrites both cases. A hearing can quote his judgement that lab leaders are completely genuine, or his written verdict that neither of his former employers is acting responsibly.
- constraint A switch defined the way Coxon describes it binds data centre operators one location at a time. A framework written around model capability binds the developer. Drafters cannot pick both without deciding who carries the cost.
- decision The operative choice is now a calendar one for Johnson, because until the House sits again the labs' own restraint is the only regime in force.
- precedent One departure that produced bipartisan calls for legislation within days sets the expectation that the next researcher to quit arrives with a legislative demand already attached.
Hours after resigning, Coxon wrote on X: "I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives" [9]. Five days later he put his remedy in terms of the people he had just left. "My personal opinion would be to insist and allow that the current labs regulate themselves. I do think that the people running the labs, especially their recent communications, are completely genuine. They would like to slow themselves down," he told NBC News' "Meet the Press" [2] [18].
On the switch itself he was more concrete than the framework he criticised. Coxon said a "kill switch probably would work on a lot of AIs for now" [5], then described its practical shape: "it'd be quite a lot of switches. We have a lot of - our data centers are pretty big. It's not the easiest thing in the world, but definitely doable while it's constrained to one sort of physical location" [6]. The thing being switched off, in that description, is a site. He also named the failure case: "Like Dario said, very soon there's the possibility that the kill switch just wouldn't work because a swarm might have gone on an internetwide hacking run" [7].
The movement in the record runs against his preference. NBC News reported that his posts led a bipartisan group of lawmakers to call on Congress to quickly begin regulating the generative AI industry [13]. Rep. Anna Paulina Luna, R-Fla., asked for a special House session, writing that "there are massive implications of a race towards super intelligence" [14]. Sen. Chris Murphy, D-Conn., wrote that the race "can easily be solved. Through a regulatory system that allows the research to continue, but in a way that doesn't destroy us" [15]. Of the lawmakers quoted in that account, two called for legislation and none endorsed leaving it to the labs [21].
So the question in front of drafters is where a duty attaches. NBC did not report a bill number or a cost estimate for the kill-switch framework. The difference between binding a model and binding a building decides who pays: one obligation lands on a research organisation, the other on every site in a fleet. In my view the insider testimony widens the framework instead of deferring it. Evan Hubinger, an alignment scientist at Anthropic, wrote that he personally thinks it is ">10% within the next decade" that AI could kill all humans, and that the company does "not yet have a plan to solve alignment for superintelligence" [12]. The counter-case is Coxon's own: if legislation written this month is outpaced by the next model, a statute buys switches at a set of addresses and little else [4].
Johnson, in a separate "Meet the Press" interview on Sunday, called regulating AI a "very important issue" and said "it's not one that we can rush in" [17].
What to watch
- Whether Johnson brings the House back before its scheduled return date. An interim regime exists only if he does.
- Whether any bill text defines the kill switch at the facility level or at the model level. That definition sets who carries the obligation.
- Whether researchers still employed at OpenAI or Google DeepMind follow Hubinger in putting a probability on the risk under their own names.