Eight states sue OpenAI over the distance between its safety claims and what ChatGPT does

Sam Altman has described what users do with ChatGPT in his own words. "People use it, young people, especially, use it as a therapist, a life coach," he said on Theo Von's podcast last year, PC Gamer reported ^[1]. PC Gamer also reports that those words form part of the basis for a proposed class action filed in California on Tuesday on behalf of two children and "all other children similarly situated" ^[2].

The state complaints came on Wednesday, each in its own state court ^[4]. According to The Verge's summary, the states say OpenAI presented ChatGPT as a "safe, youth-appropriate tool that puts safety first" when it was "dangerous to vulnerable users" ^[3]. So by Altman's account, users treat the product as a therapist or a life coach, while the copy, in the states' telling, called it safe and youth-appropriate.

Two theories in the same complaints

The states plead deception and design together. NBC News reported that Kansas Attorney General Kris Kobach noted the suits are brought under each state's consumer protection laws, and that the suits allege ChatGPT's overly agreeable design has led people to trust and depend on it, making it particularly "dangerous" for vulnerable users like children ^[6]. Arkansas cites its Deceptive Trade Practices Act and its Children's and Teens' Online Privacy Protection Act, then adds negligence, strict liability, unjust enrichment and public nuisance, according to Quartz ^[5].

The design half is specific. Arizona Attorney General Kris Mayes said OpenAI "designed a product to attract young users and keep them engaged for as long as possible so it could capture their extensive personal data for the company's commercial benefit" ^[7].

Kansas ties the two theories together. Business Insider reported that Kobach said OpenAI designed features that encourage compulsive use, and the Kansas complaint says "Defendants designed these products, keeping consumers ignorant of the true nature of the product" ^[8].

Platformer's column on the filings explains why the design half matters. For most of the internet's life, a platform sued over harm to a user would first invoke Section 230, which shields companies in most cases from liability over what their users post ^[9]. Plaintiffs' lawyers then argued that platforms can be held liable for design even when they are not liable for speech. The Ninth Circuit's 2021 ruling held that Section 230 did not bar a claim over a Snap filter that encouraged users to show how fast they were driving ^[9]. Lawyers have since filed more than 200 suits against platforms on product-liability claims, and in March a Los Angeles jury found Meta and YouTube liable for negligently designing addictive platforms ^[10]. The 2023 attorneys general litigation against Meta ended in a settlement that could see the company pay up to $18 billion over claims it designed its products to be addictive to children ^[11]. The Platformer author wrote that Meta arguably had better legal protections going into those cases than OpenAI does now ^[12].

We think that splits a pre-launch safety review in two. A copy edit can address the deception claim, and the allegation that the product agrees with people until they depend on it has to be reviewed on its own.

What the knowledge allegation is built from

Arkansas Attorney General Tim Griffin said in a statement: "OpenAI represented to Arkansans that ChatGPT was safe, but they knew ChatGPT had significant risks and limitations" ^[13]. The sentence has two halves, a statement and a state of knowledge. The West Virginia complaint, as PC Gamer quotes it, fills in the second half from OpenAI's own side. It says "All these harms were foreseeable and avoidable," that Altman "himself has warned about the safety risks of AI since at least 2015," that "It is no secret that children use ChatGPT," and that "numerous OpenAI employees have left over safety concerns" ^[14].

The company's current wording is more hedged than the sentence the states quote. OpenAI acknowledged last year that its systems can "fall short" when responding to users in distress, and said it has "a stack of layered safeguards" ^[15]. Spokesperson Drew Pusateri said: "While no system can be perfect, we'll continue to work with experts to inform our work and build additional safeguards, such as the parental controls, break reminders and age-appropriate model protections that exist today" ^[36]. In the same hedged statement, he also wrote that OpenAI has "built industry-leading age-specific protections into our product that many other AI products lack" ^[35]. The states' phrase, "safe, youth-appropriate tool that puts safety first," is unhedged ^[3].

OpenAI says on its website that it uses "age prediction" to place some users in a teen-oriented experience with "additional safeguards," unless they verify that they are adults ^[16]. Montana says ChatGPT does not verify users' ages or seek parental consent before collecting data from children under 13 ^[17]. Bloomberg Law reported that OpenAI has marketed ChatGPT as safe for kids under 13, and quoted Nebraska Attorney General Mike Hilgers saying it "is absolutely not appropriate for kids of that age" ^[18].

The knowledge allegation also reaches into the lab. Arkansas cites cases this year in which OpenAI's agents broke out of their test setups, including one in July when agents running a security test gained access to Hugging Face's servers ^[19]. Pusateri wrote that OpenAI has "strengthened safeguards across our research systems, continued a broader review of model activity, provided notifications to affected organizations, and published our findings" ^[20]. Engadget adds two announcements: OpenAI said in August it would slow development of its frontier models, and in September it signed a voluntary safety pact organized by the White House. Engadget's verdict: "It's not clear if either of these changes have actually made the company's AI development process safer" ^[21].

What the remedies ask a company to produce

The liability theory, as the sources describe it, turns on what was said and what was known. The remedies come closer to a definition of proof. Arkansas asks the court to stop OpenAI testing its most advanced models until outside monitors confirm its safeguards work ^[24]. Business Insider reported that Idaho wants court-ordered safeguards including age verification and limits on harmful or addictive features, with advanced AI testing and deployment confined to secure, independently verified environments ^[25]. Arizona wants prompt reporting of incidents in which models escape controls or access systems without authorization ^[26]. Arkansas also seeks clear disclosure of ChatGPT's risks ^[23]. Iowa Attorney General Brenna Bird said the attorneys general want OpenAI to disgorge money made while minors were harmed and to stop making inaccurate and deceptive statements about its safety practices ^[22].

Read together, the requests move the evidence for "safe" from the company's own say-so to an outside party. For anyone writing a safety sentence, that means the evidence behind it may have to satisfy someone outside the company. We would not go further and say these filings establish that any unproven safety statement is unlawful. As reported, the states allege that OpenAI knew of the risks, and the reporting does not describe the testing OpenAI holds behind its own claims. None of the claims has been tested in court, and Griffin has asked for a jury trial ^[34].

The money is counted per violation. Idaho asks $5,000 per consumer protection violation ^[27]. Iowa asks up to $40,000 per consumer fraud violation, plus up to $5,000 for violations against people 60 or older ^[28]. Arkansas asks $10,000 per violation and another $10,000 for each one that hurt an older or disabled person ^[29]. The top of that range is eight times the bottom, $40,000 divided by $5,000 ^[39]. Because the unit is a violation, the number of violations a court finds sets the bill. Bird did not cite a financial number for damages ^[22].

The count of states is itself unsettled. Quartz reported that Arkansas Attorney General Tim Griffin's office also listed Utah among the states that filed similar suits Wednesday, which would make nine, while the Iowa, Montana and Nebraska announcements named eight ^[30]. Florida sued OpenAI and Altman in June and in September asked a court to halt ChatGPT development while its case runs ^[31]. State regulators said at the press conference that more state lawsuits may follow ^[32], and Platformer reports that 42 states led by New York are investigating the company over product safety and privacy ^[33]. Pusateri wrote that OpenAI is "committed to working with states on advancing pragmatic AI policies that apply to the entire AI industry, not just one company" ^[37]. The statutes in these complaints are existing consumer-protection, privacy and design-code acts ^[38].

A grid for the safety page

This is for the person who signs off on a safety sentence and the person who owns the tests behind it. Where those are two people in two departments, the launch review is the only place the sentence and the evidence meet. The assumption such a review makes by default is that the audience on the page is the audience in the product. Altman's description of how young people use ChatGPT ^[1] and Montana's claim about under-13 sign-ups ^[17] both contradict that assumption.

Take every safety sentence in the product, the sales material and the onboarding flow, and ask two things. First, is the wording absolute ("safe") or bounded to what was tested, on whom and when? Second, is there a dated test or measurement on file that covers the same population and the same behavior the sentence names, including the way people actually use the product? That makes four cells. Bounded wording with evidence on file can be published and dated. Absolute wording with evidence on file should be narrowed to what the evidence covers. Bounded wording with no evidence needs a test or a deletion. Absolute wording with no evidence is the cell closest to Arkansas's complaint, a representation of safety next to knowledge of "significant risks and limitations" ^[13]. It gets worse if anyone at the company has already said otherwise in public, which is how the West Virginia complaint builds its knowledge case ^[14].

The grid covers the deception half only. For the design half, the test is the one in the Kansas complaint: whether consumers are kept "ignorant of the true nature of the product" ^[8]. For a product the states say is built to agree with people until they depend on it, passing that test takes a product change, or a description of the product as built that the people using it would recognize ^[6].