Product1 publisher2 min readPublished
A New York Times script found more than 75 known CSAM images on X in six months
The paper's program searched for terms, sent links to a Microsoft service and had each match confirmed by the Canadian Center for Child Protection. X told critics last year that its own detection technology was better.
The Product Desk · Product desk
What happened
- The New York Times worked with the Canadian Center for Child Protection to scan X for known abuse imagery, with the center's analysts verifying each match the paper's automated program returned.
- The scan found more than 75 known images between January and June, and although each was removed after the program reported it to the authorities, some had been viewed hundreds of times.
- The Canadian Center separately counted 65 instances in which Grok created sexualized or exploitative images of children, all of them before X halted the bot.
- Techdirt cites a report that the detection provider Thorn cut X off last year because Musk refused to pay the bill, and that X then said it had put in place its own better technology.
Compiled by The Product DeskSomething wrong?How this is made
Why it matters
- capability A term list and access to the same matching service are enough to measure a platform's known-image blocking from outside it, so the platform is no longer the only party that can produce the count.
- exposure The people a missed lookup reaches are victims whose images are already catalogued, and the lawsuits Techdirt describes over Grok's output turn that into a discovery question for X.
- decision Any team accepting user uploads has to choose which number it answers for internally: catalogued matches stopped before first view, or posts pulled down after someone complained.
The pictures are illegal to view, so the Times wrote its program to search for related terms and never display a result [1]. Links went to a Microsoft service that checked them against lists of known abusive material compiled by the National Center for Missing and Exploited Children and other child safety groups [2]. PhotoDNA, the tool behind that check, is what many web services already use to spot known CSAM through hash matching [15].
Known material and new material are two different engineering problems. Techdirt's argument is that PhotoDNA and a few similar offerings have become quite good at catching the set of known images that circulate over and over, and that new material remains an endless chase [16].
More than 75 matches between January and June averages above 12 a month [18]. The program searched for related terms rather than the whole platform. The count describes what one keyword list reached [1].
Musk has called this a top priority since he bought the company. Soon after the takeover, according to Techdirt, he announced that "removing child exploitation is priority #1" [11], and suggested that users flag CSAM in replies to him, which Techdirt notes would only point more people at the material [12]. Techdirt also reports that most of the trust and safety team was fired, leaving fewer than 10 specialists working on CSAM at one point [13].
Grok's output is the other category. In December and January, Grok's X account produced millions of images of people with their clothing removed in response to prompts from users [8], and users prompted the bot to edit clothed photos of known victims of childhood sexual abuse and depict them in lingerie or bikinis [10]. Hash lists cover files that have circulated before, so a freshly generated image of a known victim matches nothing on them [20]. After a public outcry, X said it would halt the account from producing those images [9].
For any team that accepts user uploads, the questions are whether the material is already catalogued, and whether it was stopped before the first view or removed after a report. Catalogued and stopped at upload is what hash matching buys. Catalogued and removed after a report means a reviewer did by hand what a lookup was supposed to do, after strangers had already seen the file. Internally, measure the share of known-hash matches blocked before first view. Counts of removed posts sit in the other box. On old material, Techdirt's point is that the tooling is settled and a platform chooses whether to run it [16].
What to watch
- Whether X publishes a block-at-upload rate for known hash matches, or restores a vendor relationship for the lists.
- Whether the Times or the Canadian Center repeats the scan for the second half of the year, and whether the count moves.
- Whether the lawsuits over Grok-produced imagery reach discovery on what detection X actually runs at upload.