Product1 publisher2 min readPublished
Contractors grading ChatGPT replies see the whole chat and a summary of the user's memories
Leaked documents reported by 404 Media describe a review program codenamed Project Lily that pays hundreds of contractors over $50 an hour to read real conversations. Model training is on by default for Free, Plus and Pro accounts.
The Product Desk · Product desk
What happened
- 404 Media reports that OpenAI pays hundreds of outside contractors, often more than $50 an hour through third-party staffing firms, to read, summarize and critique real ChatGPT conversations.
- Under the internal codename Project Lily, a contractor reads the user's prompt, summarizes what the user wanted, and scores four candidate responses on a 1-to-7 scale.
- OpenAI says reviewers are never given usernames and that every conversation first passes through an automated Privacy Filter built to redact identifying details.
- Model improvement is on by default on Free, Plus and Pro accounts, and off automatically on Enterprise, Business and Edu accounts.
Compiled by The Product DeskSomething wrong?How this is made
Why it matters
- exposure Because reviewers see whole conversations, a thread that opens with a recipe and drifts into a client's contract terms arrives on a contractor's screen as one document, with nothing splitting the personal part from the commercial one.
- constraint The toggle limits training use, and automated Terms of Service and safety scanning continues regardless, so nobody can honestly tell staff that switching it off means no system reads their chats.
- cost No setting retrieves whatever staff pasted in before the switch: it is already de-identified and queued in the training cycle, so the cost of the last quarter's habits is sunk.
- decision Anthropic and Google run human review on the same conditional basis. So a team writing a ChatGPT rule also has to decide whether Claude and Gemini get the same treatment or stay on their shipped defaults.
Among the prompts that reached reviewer screens were ones where the user asked ChatGPT to "keep this between us," according to 404 Media's reporting as relayed by Tom's Guide [11]. That instruction is addressed to the model, and it does not override the platform's backend data collection [19].
The guides tell reviewers what to punish. Unnecessary checkmark emojis, engagement-bait endings and cluttered formatting all lose points [5]. The model may not claim human lived experience, and "As a chef, I like to..." is strictly banned [6]. Answers that excessively flatter the user, validate irrational thoughts or amplify personal frustration get flagged [7]. Someone is paid over $50 an hour, through a third-party staffing firm, to judge whether ChatGPT sounded sycophantic in a stranger's conversation about a bad week [3].
Take the smallest number "hundreds" can mean, 200, and that reported $50 floor. The reading costs at least $10,000 an hour in contractor pay [18].
OpenAI acknowledges on its own site that the Privacy Filter can make mistakes, missing uncommon identifiers or ambiguous personal context [9]. A separate block on the reviewer's dashboard tells the reviewer more about who the user is than a name field would. 404 Media found that the dashboard often includes a "user memories summary," an overview of the user's previous interactions that can reveal their general location, profession or personal life context [10]. The redaction pass runs on the conversation in front of the reviewer, and the memory digest was assembled from the ones before it.
The Tom's Guide account does not say how often work or customer data turns up in reviewed conversations [20]. It does report who reads them, and which accounts are opted in by default.
So sort your accounts on two questions rather than one. Do you administer the account, and does anyone else's data go into it. Accounts you administer are a procurement matter, settled when the seats are bought. Accounts you do not administer, carrying only the employee's own material, are the employee's business. The quadrant that costs you is the personal Plus account with a customer's email pasted into it for a rewrite: the only lever there is the person opening Settings, selecting Data Controls, and switching off "Improve the model for everyone" [14].
Which makes the honest version of the Monday message narrower than most policies are written. You are not telling staff that their chats are private. You are telling them which tier they are on, where the toggle lives, and that a contractor being paid to fix the model's tone is the audience for anything they type into the wrong account [1].
What to watch
- Whether OpenAI publishes a Project Lily headcount, the staffing firms involved, or the volume of conversations reviewed.