Security1 publisher2 min readPublished
California prices negligent platform design at up to $1 million per child
Gavin Newsom signed a package Thursday that also bars addictive feeds for anyone under 16 and requires AI chatbot operators to assess risk before rollout. Meta says the feed rule hurts what it offers teens.
The Watch · Security desk
What happened
- Newsom signed a package of child-safety technology laws Thursday that penalize large social media companies up to $1 million per child if they are found negligent of harming children through their platforms.
- One of the laws bans tech businesses from offering addictive feeds to anyone under 16.
- Another requires operators of AI chatbots to perform risk assessments before rolling them out.
- The signings follow Meta's settlement with California and 28 other states over a 2023 suit alleging it deliberately designed features to addict children, under which it agreed to pay up to $18 billion.
Compiled by The WatchSomething wrong?How this is made
Why it matters
- exposure Child-safety design becomes a reserve line: one finding that covers a thousand minors reaches a billion dollars at the ceiling.
- constraint A ban that applies to anyone under 16 only bites on accounts whose age the platform has determined, so age classification becomes the control every ranking decision depends on.
- decision Chatbot operators have to decide whether the pre-deployment risk assessment is a document filed after the fact or a gate with authority to stop a launch.
- precedent A penalty fixed per child in statute gives other legislatures a schedule to copy, instead of leaving child-safety enforcement to negotiated multistate settlements after the harm.
The account of the signings does not carry bill numbers or effective dates. It leaves out what makes a design negligent and how a platform is expected to establish that a user is under 16 [18].
The per-child ceiling is the figure that moves budgets. At $1 million per child, a finding covering 1,000 minors reaches $1 billion [2][16]. For scale, the ceiling on Meta's settlement with California and 28 other states is the same number as the per-child maximum applied to 18,000 children [7][17]. That settlement figure came out of a multistate action against one company [7].
Meta objected to the feed restriction, saying the law hinders its ability to bring enjoyable content to users on its platforms [8]. Spokesperson Jim Cullinan said personalization "is also how we deliver age-appropriate content for teens that is relevant to them" [9].
Newsom's record on chatbots runs in two directions. He signed a law last year requiring platforms to remind users they are interacting with a chatbot and not a human [10]. Hours later he vetoed a bill that would have banned companies from making AI chatbots available to anyone under 18 unless the businesses could ensure the technology could not engage in sexual conversations or encourage self-harm [11]. In 2024 he signed a law restricting student use of smartphones in schools [12].
A separate track moved a day earlier. The laws signed Wednesday, which Anthropic supported, require the state to create rules for independent organizations to evaluate the safety risks of AI models and to create a registry of AI auditors who would have to be financially independent from the businesses they audit [14].
Assemblymember Rebecca Bauer-Kahan said, "We have designed cribs to be safe, chairs to be safe, car seats," and "Yet we've allowed technology to be handed to our children and never asked or expected it to be safe" [6]. Assemblymember Josh Lowenthal, a Democrat who helped write the law imposing the penalties, said "The pace of technology is moving faster than government can put guardrails in" [13]. Newsom, who signed the package alongside his wife and state lawmakers at a children's museum near San Francisco, said "It's a good day for our children" [5][15].
What to watch
- Bill numbers and effective dates, and whether the penalty accrues per child per violation or per child overall.
- The first enforcement action, which is where a regulator has to define negligent design.
- Whether Meta, which objected to the feed restriction, challenges it in court.