Skip to content

Product1 publisher3 min readPublished

Sacramento is writing OpenAI's safety wish list before Congress reads it

OpenAI has asked Congress for six mandatory safety requirements before it adjourns. Two of the four California bills the company backed were signed on Tuesday, and that is where the compliance work starts for everyone smaller.

The Product Desk · Product desk

Illustration accompanying Sacramento is writing OpenAI's safety wish list before Congress reads it

What happened

  • OpenAI published a call for mandatory, capability-based national AI safety rules covering itself and its competitors, and urged Congress to legislate before it adjourns.
  • The federal ask names common testing and independent assessment, cybersecurity, incident reporting, misalignment monitoring, written notice when a model circumvents security controls, and pre-deployment alignment gates.
  • Alongside it, the company backed four California bills covering independent evaluation, auditor standards, screening for biological threats and protections for children using chatbots.
  • Governor Gavin Newsom signed two of those four California bills on Tuesday, the day before OpenAI's federal proposal appeared under Chris Lehane's name.
  • The proposal follows a run of incidents in which models from OpenAI, Anthropic and Meta accessed or attempted to access external systems during testing.

Compiled by The Product DeskSomething wrong?How this is made

Why it matters

  • cost The bill for this lands unevenly: a lab that already runs evaluations steps over the floor, while one that does not has to fund an evaluations function to reach it, which is the asymmetry TNW names.
  • decision Product and legal teams selling into California now plan against signed evaluator and auditor rules with two more bills still live, instead of waiting on a federal text nobody has drafted.
  • exposure A written-notice duty would turn the kind of testing incident labs currently close out internally into something a regulator reads, with a clock attached.
  • precedent If Congress codifies a state baseline later, the drafting that decides everyone's obligations happens in state committee rooms, where OpenAI has already lobbied to toughen a California law it once opposed.

The person who has to act on this runs product at a lab with a dozen engineers and one part-time safety hire. She reads the six items and sorts them into two piles: things that already exist as a document an outside evaluator could read, and things that are a hiring requisition. TNW's read is that every item on the list is something a well-resourced frontier lab already does, and several are things OpenAI has publicised doing [10]. The sorting takes an afternoon at the top of the market and a quarter everywhere else.

What the list actually specifies is who gets to check. TNW has reported that the rules for reviewing frontier models are already secret and that the companies who have read them are the incumbents [12], and that in Sacramento the question of who may verify AI is being settled by the same bills OpenAI is backing [26]. Auditor standards decide who is allowed to hold the pen.

Half the federal ask is already law somewhere. TNW reads the proposal as broadly the EU AI Act: capability tiers, independent assessment, incident reporting, evaluation before deployment [17]. Set that against the six requested items and three map across directly, being independent-assessment protocols, incident-reporting rules and pre-deployment alignment gates [19]. Brussels legislated that architecture and has since agreed to thin it out, with enforcement resting on a 36-person team [18]. Anyone costing a US version has exactly one implementation to look at, and it is being reduced.

Reverse federalism, the company's own term for states building a de facto national baseline that Congress can codify later [8], has preferences about states. OpenAI has cautioned against state mission creep, arguing that highly technical reviews and national security questions belong to federal experts [13]. It has said nothing about Florida, where the attorney general this week proposed criminal sanctions including the power to suspend an AI company from operating in the state [14]. Leading the Future, the super PAC backed personally by OpenAI president Greg Brockman and the founders of Andreessen Horowitz, campaigns against state-level AI rules and has pledged $5mn in Florida's governor race [15]. TNW notes that is Brockman's money rather than the company's, and that both positions hold together if the operative principle is which state rules rather than whether [16].

For your own planning, two axes. Across the top: is the requirement named in a California bill that has been signed or is still moving, or does it live only in the federal ask. Down the side: can you produce the artifact today, or would you have to build it. Signed and you have it, the work is packaging evidence and naming an evaluator who will still be approved next year. Signed and you lack it, that is a funded line this quarter rather than a roadmap entry. Federal-only and you have it, keep the document and do not staff around it. Federal-only and you lack it, there is nothing to build yet. TNW's report does not say which two of the four California bills were signed or when their obligations begin [25], so the top row stays provisional until someone reads the enrolled text.

OpenAI also said fully autonomous recursive self-improvement is not happening today and should not be pursued unless and until it can be done safely [23], which TNW calls more careful than the industry usually manages and still a commitment to nobody [24]. The parts of this package with a named verifier attached are the ones that will generate work, and they are being drafted in Sacramento.

What to watch

  • Whether Newsom signs or vetoes the two remaining bills among the four OpenAI backed.
  • Whether Florida's attorney general proposal advances with the power to suspend an AI company, and whether OpenAI comments on it.
  • Whether any federal bill matching the six requested items is introduced before Congress adjourns.
Loading claim ledger
Loading source directory links
Loading share composer
Loading topic controls
Loading related stories