Product3 publishers3 min readPublished
Newsom gives a working group two months to redraw California's frontier-model audit rules
Newsom's executive order asks an expert group he has not yet named for recommendations on onsite verifiers, independently audited risk reports and a routinely tested kill switch. California has until January 2028 to certify anyone to do the testing.
The Product Desk · Product desk
What happened
- Newsom signed an executive order on Friday telling a working group to report within two months on strengthening California's AI safety laws, including whether companies should build a kill switch for frontier models.
- Newsom vetoed SB 1047 on 29 September 2024, a bill requiring developers of covered models to promptly enact a full shutdown, saying it regulated models by size and cost instead of by use.
- CDT's Travis Hall has asked the governor to include advocates for civil rights and liberties in any working group set up for these processes.
Compiled by The Product DeskSomething wrong?How this is made
Why it matters
- decision A team shipping or fine-tuning frontier models has a planning date before it has a rule, so the defensible work is assembling audit evidence: which named person can halt a training run, and what the incident log calls a loss-of-control event.
- cost On the only worked example so far, the company being tested met the tester's compute bill. Third-party evaluation sat on the developer's budget line, not the state's.
- constraint California has until January 2028 to certify organisations able to test frontier models, so any requirement resting on certified testers is limited by tester capacity before it is limited by statute.
- contradiction Europe put the power to restrict, withdraw or recall a general-purpose model with the regulator, while California is weighing a switch built by the developer, so a company in both markets prepares for two different failure modes.
The order sets one hard date, and it is for a document. A working group has two months to recommend how to strengthen California's AI safety laws [1][2]. "We're going to do this thoughtfully but with urgent velocity; the stakes are too high to wait or delay action," Newsom said in a statement [19]. His office has not said who is in the group [7].
The brief is the part a product owner can act on now. According to The Verge, the group is told to consider requiring AI companies to embed independent verification groups onsite for regular audits. It is also to consider holding transparency reports and risk assessments to the standards of independent auditors, and creating a kill switch that is routinely verified as effective. And it is to consider requiring companies to report "loss-of-control incidents" like the OpenAI attack on Hugging Face as critical safety incidents [5]. Three of those four items are record-keeping and access; the fourth is a build.
A first audit would ask for the name of the person who can stop a training run or pull a deployment, and the authority they do it under. Most labs already have their own means to halt rogue systems, governed by humans and in-house safety and alignment teams, and no US law requires them to, Mashable reports [18].
Newsom has been here before. He vetoed SB 1047 on 29 September 2024, a bill that would have obliged developers of covered models to "promptly enact a full shutdown", stopping operation and further training. He argued it regulated models by size and cost instead of by use [8]. This order arrives 11 days short of the two-year anniversary of that veto [23]. In a video on his X account, Newsom called the measure an emergency shut off, "otherwise known as an 'AI kill switch.'" He said the term "means a lot of things, depending on who you talk to" [4].
The people who would have to define it disagree about whether it can be defined. Mark Nitzberg, executive director of the University of California, Berkeley Center for Human-Compatible AI, told Scientific American that a universal AI kill switch is "more of an aspiration than a reality" [14]. Travis Hall, Director for State Engagement at the Center for Democracy and Technology, said the order "rightly prioritizes the development of better guidance regarding the definitions and requirements for AI safety regulations, particularly in regards to third party auditing" [16]. Hall also said CDT remains "concerned that proposals for 'kill switches' will prove ineffective in reducing risk while introducing new security risks and serious potential for government abuse" [17].
For teams selling into the EU, the sequencing is different. The AI Act's stop button applies to a human overseeing a deployed high-risk system and not to the model underneath, and those obligations are not yet in application [13].
One question is whether a rule of this shape would reach you, meaning you develop or fine-tune a model California would treat as frontier. The other is whether your incident log already separates a loss-of-control event from an ordinary outage. Answer yes to both and the next quarter is documentation. If you are in scope and the log does not make that split, the two-month report is the calendar to plan against. The auditing track is already law: the order tells a state agency to speed up the independent verifier framework and the state registry of AI auditors that Newsom signed [6]. SB 813's deadline for certifying organisations able to test frontier models falls roughly 14 months after those recommendations are due [24].
What to watch
- Whether Newsom calls the special legislative session on AI he floated to Politico. That session is how the recommendations would become bills.
- The names on the working group, and whether civil rights and liberties advocates get seats alongside technical experts as CDT asked.
- Whether the accelerated verifier framework and auditor registry produce any certified frontier-model testers ahead of SB 813's January 2028 deadline.