Product2 publishers3 min readPublished
Meta fixes the AI suggestion that asked a mother to identify her own child
Meta AI asked a mother to name the child in her own video, then, she says, pulled her daughters' ages and location out of years of family posts. Meta says the answer used only data she could already reach.
The Product Desk · Product desk
What happened
- Instagram user Kalie Robins cross-posted a clip of herself and her child singing in the car to Facebook, and Meta AI suggested the prompt "Who is the child passenger?" beneath it.
- Meta spokesperson Dina El-Kassaby told The Verge the company "missed the mark" and that the feature never should have prompted the user with questions like that.
- Meta says the issue is fixed, and it has not detailed what changed in the prompt system or how often comparable suggestions were generated beforehand.
Compiled by The Product DeskSomething wrong?How this is made
Why it matters
- constraint Meta's defence is that the response used only data the account could already reach, so the permission model was the entire scope on the feature. Any team building suggestions over existing user content inherits that same single guardrail until it writes a subject rule of its own.
- exposure Per Robins, part of the material came from relatives' posts, so the people described by the feature were not the ones using it.
- decision Anyone shipping a generated prompt surface now has a documented case to cite when arguing for a list of subjects the system may not raise on its own, reviewed before launch.
- precedent This is the second Meta AI feature altered or withdrawn following public complaint, after July's Instagram deepfake tool. Retraction after launch is the visible remediation route at Meta.
El-Kassaby's statement to The Verge concedes the prompt and defends the data access. "While the content was available to this user, these prompts should never have been generated," she said [14]. Both halves can hold at once, and the combination is the useful part for anyone building a suggestion layer over a corpus of user content that already exists. Account permissions scoped what the assistant could fetch, and nothing in that model scoped what the product was willing to ask about.
Meta's stated purpose for the feature is topic discovery. "We created this feature to help people get more information about topics or posts that interest them," El-Kassaby said [7]. That works when the post is about sourdough. Applied to a clip of a parent and a child singing in a car, the generator had one subject available to be curious about, and it asked who the child was [2].
Teams building these surfaces treat a suggested prompt as a shortcut into content the user already owns. Users read it as the product speaking to them about the thing on screen. Robins says she tapped it, and the assistant returned details about her daughters drawn from her own earlier posts and from posts by her relatives, then offered follow-up prompts about the children's ages and where the family lives [3][4]. One prompt under one video became three prompts about the same child inside a single session [18].
The deleted-photo part is Robins' account, and it should be read as hers: she says the assistant showed photos of her daughters including one she had removed years earlier [5]. Dexerto made the wider point that deleting one post is not the same as removing every copy or reference to that information elsewhere on social media [15]. Meta's own explanation points at a second surface. El-Kassaby said the assistant may have pulled from videos in which Robins named her children [9].
The scoping test worth running before a generated surface ships has two axes, and permissions only cover one of them. Axis one: can the system retrieve the item for this account. Axis two: did the user ask for it. The quadrant this feature landed in is retrievable plus system-initiated, the only one of the four where the product picks the subject. Anything landing there needs a written rule about which subjects the system may raise unprompted, and minors, health, location and family relationships are the obvious four to name in it.
The cheap version of that check is to take a week of prompts the generator produced against real accounts and read them out as sentences an employee would say to the account holder's face. "Who is the child passenger?" fails that reading whether or not the video was public, which is what El-Kassaby's "missed the mark" concedes [6]. What Meta changed in the prompt system, and how many similar suggestions were generated before the fix, remain unpublished [13]. It pulled a different AI feature in July, the one that let people create deepfakes of other Instagram users, after what The Verge described as fierce backlash [12].
What to watch
- Whether Meta publishes what it changed in the prompt generator, or a count of how many similar suggestions fired before the fix.
- Whether the same class of suggested prompt shows up in Messenger and WhatsApp, where Meta has also embedded the assistant.
- Whether other users document the same cross-account assembly; Robins' session is the only one described in these reports.