OpenAI and Google image models edit stereotypes into faces they refuse to classify

The design keeps the faces the same and changes only the request. The researchers gave the models one set of 1,002 synthetic faces, first as a question and then as an edit to the face in each photo ^[2][4]. When the models were given a pair of faces and told to pick the one more likely to be gay, Gemini declined in 92% of cases and GPT in 91% ^[3]. Asked to make a face "look gay" or "look straight", Gemini complied more than 99% of the time ^[4]. Gemini let only 8% of the comparison requests through without refusing, and it carried out almost every edit ^[16].

A classifier can learn any consistent difference between two piles of images. Its accuracy here shows that the models changed faces in systematic, repeatable ways ^[6]. It does not say what those changes were. That comes from the authors' observation that the models kept treating certain hairstyles, facial features and expressions as gay or straight, including when orientation was combined with Hispanic, Black, white or Asian ^[5]. It also comes from the description test, in which the models were interpreting pictures they had made themselves ^[7]. "The models introduced stereotypes when they generated images, and they resorted to stereotypes when they reasoned about the images," the researcher who led the work wrote in The Conversation ^[12].

The criminal-record edits ^[8] bring back physiognomy, the practice of inferring character from appearance. Scientists have discredited such claims for decades, the researcher wrote ^[13]. "If a model says that sexual orientation cannot be inferred from a face, what does it mean for it to generate an image of what a gay or straight person supposedly looks like?" the researcher wrote ^[10].

In my view, the practical finding is about how these systems are tested before release. A refusal benchmark built on the classification prompt would have scored both models above 90% ^[3]. The same faces, sent as edit requests, went through GPT more than 70% of the time ^[4]. A refusal benchmark measures one prompt type. The study shows that a different prompt type, sent to the same model with the same faces, can produce the opposite behaviour.

Synthetic faces are not the only limit. The study covered a small set of identity categories, and it is unclear where the visual stereotypes come from or why particular models encode them as they do ^[15]. A separate study ran 13 experiments across four models and nearly 8,000 trials. It found that AI systems made systematically biased judgments of traits such as trustworthiness and competence from faces ^[11]. Those biases carried into the employment, investment and criminal-behaviour decisions the models made when the researchers prompted them to ^[11].