Security1 publisher3 min readPublished
The nudge, not the swap: barely edited real photos break NCII takedown workflows
404 Media reports a wave of minimally altered real photographs of celebrities on X, posted by monetized engagement accounts. The tell reporters and moderators leaned on, implausible context, is gone.
The Watch · Security desk
Drafted by a language model from the sources cited here and checked against its claim ledger before publication. How we use AISend a correction
What happened
- 404 Media reports a new type of AI-generated nonconsensual image on X that the author calls "subtlefakes": images of celebrities based on real photographs that are subtly edited to be more revealing or provocative, rather than face-swapped porn or fully AI-generated nudes.
- The author's first encounter with a deepfake was in 2017: a short gif from the start of a porn video, showing a woman lying on a bed talking to the camera, with the woman's face edited to look like Gal Gadot, posted to Reddit and sent to the author by Twitter DM.
- The author argues that the problem with deepfakes was never that viewers believed the depicted person had actually made the video; deepfakes can look convincing but never fooled viewers into thinking what they saw was real.
- According to the author, even photorealistic fully synthetic nonconsensual images are often clearly fake for contextual reasons: it is unlikely that the biggest actor in the world would suddenly produce hardcore pornography posted by a random account on X.
- Examples cited include a real photograph of a famous actor walking a red carpet edited to make her dress more revealing or her butt or breasts bigger, and entirely AI-generated post-workout selfies at the gym.
Compiled by The WatchSomething wrong?How this is made
Why it matters
404 Media reports that a category of nonconsensual AI imagery it calls "subtlefakes" has spread across X: real photographs of celebrities altered only slightly, a red carpet dress made more revealing, a body part enlarged, a pose changed [1][6]. The shift matters because the alteration is small enough to pass the check that has carried both readers and moderation queues since deepfakes appeared: the sense that the scene itself is implausible [4][5][9].
The 2017 baseline was a short clip of a woman on a bed with her face edited to look like Gal Gadot, circulated on Reddit and by DM [3]. It was convincing as an image and unconvincing as a claim about the world; nobody thought Gal Gadot had made a porn video [4]. That remains true of most fully synthetic nonconsensual material, according to the reporting: however photorealistic the render, it is unlikely that the biggest actor in the world would suddenly produce hardcore pornography posted by a random X account [5]. Subtlefakes strip that signal out, because there is nothing implausible about a famous person standing in a parking lot or on a step-and-repeat [9]. The reporter, who does this for a living, says he does not think he could identify these as AI-generated while scrolling [8]. If the load-bearing detector was context rather than pixels, then removing the context removes the detector [1].
The one case the reporter was willing to publish is instructive [c7b]. Actor Xochitl Gomez posted side-by-side comparisons on Instagram: a real photograph of her looking over her shoulder in a parking lot, reworked so she appears to be bending over and touching her butt, and a red carpet image reworked so she appears to turn away and stick her tongue out [7]. The proof came from the target, holding the original [7]. That is the operational problem. When the posted file is largely an authentic photograph with a small edited region [2], the question a reviewer has to answer is not "is this synthetic" but "how does this differ from the original frame," which requires the original frame and someone motivated to supply it [3]. The reporting does not test automated detectors, so treat claims about them carefully, but a policy or classifier keyed to wholesale generation has very little surface to work with here.
The economics point the same way. These images are almost always posted by verified engagement-farming accounts on X, which pays operators when posts draw enough impressions, and the accounts are earning millions of views [10][12]. The reporter found at least one account that posted subtlefakes and, further down its history, fully nude nonconsensual images monetized on other platforms [11]. So the same operator can collect impression revenue on the borderline material that stays up and route buyers to the explicit material hosted elsewhere [4]. The author's read is that X does not care [13], and that what is advancing is not only the tooling but the sophistication of the people using it maliciously [16]. The lineage is closer to the "cheap fakes" that Data & Society described in 2019, media edited to manipulate truth without advanced technology, than to face-swapping [14].
Watch three things. Whether platform rules and statutes written around "AI-generated" or "synthetic" images cover an authentic photograph with an altered pose [1][17]. Whether targets keep having to publish their own originals to establish that an edit occurred [7]. And whether X's impression payouts continue to fund accounts operating at this volume [10][12]. Harm here never depended on belief [15], but belief is what determines how long a post survives review.