Leadership1 publisher2 min readPublished
The consent fight over AI faces is starting inside families
Relatives are putting children's faces into image generators out of pure affection, and the automated filters, user lawsuits and proposed age limits now on offer were all built for a different kind of uploader.
The Board Room · Leadership desk
What happened
- Ben Metcalfe, who works in AI governance, visited his grandparents and found an AI-generated caricature of one of their family photos hanging on the kitchen wall.
- Jen, a 32-year-old mother, says her own mother used Grok to turn photos of her two toddlers into images of them on Santa's lap and riding the Polar Express, then texted them over last Christmas.
- xAI faces a lawsuit from a child sexual abuse survivor alleging Grok created new deepfake exploitive images of her after real images of her abuse were uploaded; xAI did not respond to a request for comment.
- Meta shut down its Muse Image tool, which let users generate images from other people's publicly posted photos, less than a week after launching it this summer amid backlash.
- Europe is considering regulations that would limit chatbot and AI companion access to under-15s, while lawmakers press to remove children from social media altogether.
Compiled by The Board RoomSomething wrong?How this is made
Why it matters
- constraint A likeness rule aimed at criminal misuse leaves the ordinary case unmanaged, so the only control available is one relative persuading another, one upload at a time.
- exposure The person in the picture never transacted with the vendor. Consent sits with whoever holds the file, and the subject has nothing to withdraw.
- cost Meta paid for treating a public post as permission within days of shipping, and the bill fell on the product owner.
- precedent Normalisation by visibility moves the default toward permission. Each unchallenged upload makes the next one easier and makes a later objection look like the unusual act.
Whoever holds the photo does the consenting. Ben Metcalfe, who works in AI governance, said of the caricature in his grandparents' kitchen: "I understand that there was good intention behind doing it," and, "At the same time, it's a major privacy concern." Jen, whose mother made the Santa pictures, said: "It's so cute. And I want to love it."
Her account names three uploaders inside one family circle. Her mother at Christmas; a friend who sent a coloring sheet printed with her kids' faces for Mother's Day; and her brother-in-law, who used AI to add one of her children to a family photo where a stroller had covered the child, napping, when the shutter went. None of the three meant any harm. "We all aren't educated enough about what AI companies are doing with our content," Jen said. Business Insider changed her name to avoid sparking more family drama.
The remedies now in play are aimed at criminal misuse. X told The New York Times this month that it blocks users who post or engage with such material, reports them to authorities, and uses automated filters to block exploitative material from being uploaded. xAI has sued users who face criminal charges over allegedly using Grok to create explicit images of children. Elon Musk called the exploitation of children on X "priority #1" when he bought the platform in 2022.
Meta's Muse Image is the one case here of a company shipping the insider problem as a feature, by letting users generate images from photos other people had posted publicly. It lasted under a week. The personalisation trade runs the other way, selling eCards, toys, T-shirts, tchotchkes and children's books built from uploaded family pictures.
Metcalfe said the spread is social. "It's this effect of, well if they're doing it, then we can do it as well, and then it becomes a chain reaction of everyone else is kind of doing it," he said. Six generations now share the same platforms, from the Silent Generation to Gen Alpha.
Nothing in this account is about a workplace. The relation is the same one an employer has to a client's photograph or a colleague's headshot: the person who clicks agree is somebody else, and the person in the frame is never asked. A rule about whose likeness may be uploaded, and who has to be asked first, outlasts the next model release. Jen said she isn't even sure what boundaries are best to set around how loved ones use her children's images.
What to watch
- Whether Europe's under-15 proposal ends up addressing the uploading of a third party's image, or only a minor's own access to chatbots.
- Whether the xAI lawsuit produces a ruling on liability for what a model generates from images somebody else uploaded.
- Whether any consumer image generator begins asking the uploader to attest that the person depicted agreed.