Skip to content

Product1 publisher3 min readPublished

The EU's draft Kids Act would strip companion chatbots of emotion and memory for under-18s

A draft seen by WIRED would bar chatbots from simulating emotions with anyone under 18 and stop them carrying memory between conversations. Fines reach 6% of worldwide turnover. The scoping work lands on product teams.

The Product Desk · Product desk

Illustration accompanying The EU's draft Kids Act would strip companion chatbots of emotion and memory for under-18s

What happened

  • The EU Kids Act, proposed on Thursday, would ban chatbots from simulating emotions or interpersonal relationships in interactions with anyone under the age of 18.
  • Chatbots operating in the EU would also not be allowed to retain memory of interactions between separate conversations, according to a draft of the legislation seen by WIRED.
  • Children under 13 would be barred from accessing chatbots at all without their guardians' supervision.
  • Platforms that do not build child-safety protections into their products face fines of up to 6% of total worldwide turnover.
  • The proposal covers AI companions and general conversational chatbots that the Commission sees as posing serious risks to minors' health, safety and well-being.

Compiled by The Product DeskSomething wrong?How this is made

Why it matters

  • constraint Cross-session memory becomes a switch that has to be conditional on who is signed in, and the profile and retrieval layer needs a session-scoped mode that most products were never built to offer.
  • decision Every surface a minor can sign into needs a classification call before any design work starts, because the draft reaches only the companion and general-conversational category.
  • exposure A penalty pegged to total worldwide turnover moves a teen-facing feature onto the group risk register, in front of approvers who have never opened the settings screen it lives on.
  • precedent Australia and the UK are working at minors' chatbot access through their social media bans, so a team building age gating for Brussels is designing for three regimes at once.

The draft goes at the two features that make a companion product work. One is continuity: the bot knows what you told it last Tuesday [4]. The other is emotional register: it responds as though it has feelings about you [1]. Around half of Europeans aged 11 to 25 have discussed intimate or personal matters with an AI chatbot, according to an Ipsos BVA survey across France, Germany, Sweden and Ireland [7]. Seven of the fifteen single-year cohorts in that band sit under 18 and inside the proposal's scope [8].

The Commission's stated reason for the memory rule is data accumulation. It worries that systems will build up sensitive data about minors over time and reinforce harmful interaction patterns [5]. It also says dependency may hurt a child's development and their relationships with other people, and that chatbots give minors dangerous advice on mental health and personal development [14]. "Platforms have been playing fast and loose with children's mental health," von der Leyen said at a press conference on Thursday [11]. As WIRED describes the memory rule, it applies to chatbots operating in the EU, not only to accounts held by minors [4].

Inside teams, memory is the thing that removes repeated setup and makes an assistant useful in week four instead of week one. The draft treats it as a file that keeps growing on a 14-year-old. The one outcome measure in the Commission's case runs against the feature. Across 91 countries, children who say they use AI chatbots for schoolwork get lower test scores than children who do not, according to OECD research [9]. The OECD figure compares two groups of children; it does not show what causes what.

The proposal also goes after defaults. Social networking sites, video-sharing platforms and games should not automatically activate or encourage AI chatbots, and should not pin AI companions or general conversational bots at the top of a minor's contacts list [10]. Snapchat had done that [10].

Both age rules turn on knowing who is 12, who is 17 and who is 19. WIRED's account of the draft does not say how a platform is meant to establish that. The determination falls to whatever age signal a team already collects, and the build follows from it: a session-scoped mode with memory writes disabled, plus a response policy that strips first-person affect, applied to every account the system believes belongs to a minor.

The scoping call belongs to each surface separately, and it has two parts: can someone under 18 reach it, and does it hold state between sessions or talk about its own feelings. An order-status bot that forgets you answers no twice and changes nothing; a companion answers yes twice. The hard case is the general assistant with memory on by default that a 15-year-old can sign into, because the draft only reaches chatbots the Commission counts as posing serious risks to minors [6].

Simeon de Brouwer, a policy adviser at the Brussels-based advocacy group European Digital Rights, welcomed the chatbot measures but told WIRED that "it doesn't make sense to restrict them to young people only" [12]. The EU says its Digital Fairness Act, due later this year, is expected to propose clamp downs on addictive design more widely [13].

What to watch

  • Whether the final text limits the memory ban to minors' sessions or keeps it as a blanket rule for chatbots operating in the EU.
  • Whether the Commission publishes criteria for which chatbots it counts as posing serious risks, since that classification decides who has to rebuild anything.
  • Whether the deepfake concern von der Leyen raised in the State of the Union turns into an AI Act change, given that labeling has applied only to commercial deepfakes since August.
Loading claim ledger
Loading source directory links
Loading share composer
Loading topic controls
Loading related stories