The Pulse
OpenAI Contractors Read Real ChatGPT Conversations, Report Says
A 404 Media investigation based on leaked documents and anonymous sourcing says contractors review ChatGPT conversations to help assess responses.
AI.info Team ·
More than 900 million users, and some chats reach human reviewers
OpenAI is using hundreds of contractors to read real ChatGPT prompts and assess the chatbot’s replies, according to a secondary investigation published by 404 Media on September 14, 2026. The report is based on leaked internal documents and anonymous sourcing. A contractor told 404 Media that Mercor ultimately pays the reviewers. The prompts can include complete conversations and sensitive personal information, even though many users may assume they are communicating only with software.
The contractors work under an internal initiative called Project Lily. Their assignment is to improve how ChatGPT responds by summarizing what a user wants, comparing several generated answers, assigning scores, and explaining which parts of each response work or fail.
OpenAI’s chatbot has more than 900 million users, according to the figure cited by 404 Media. The scale of that user base means the company’s review process draws from a large and varied stream of everyday conversations, including requests involving personal relationships, work, health, and emotional support.
Project Lily turns private prompts into training tasks
Documents reviewed by 404 Media describe a three-stage workflow. A reviewer first reads the user’s prompt, then writes a short explanation of what the person is trying to accomplish, and finally rates a set of ChatGPT responses. The reviewer must identify specific strengths or weaknesses and provide a written rationale.
The rating system runs from one to seven. A score of one means a response is “unacceptable, unusable,” while seven means it “would be hard to meaningfully improve.” Useful information can still receive a low score if the answer is excessively long, cluttered, poorly matched to the user’s tone, or filled with stylistic habits that reduce its usefulness.
Project instructions tell reviewers to flag sycophancy, forced imitation of a user’s style, patronizing assumptions, emotional escalation, and endings designed to keep users engaged. The documents also direct reviewers to penalize “AI-speak” and inappropriate emoji use, while asking ChatGPT to sound natural without presenting itself as a human being with personal experiences.
“No,” someone who works with the prompts said when asked if they think ChatGPT users know that humans are reading their chats. “I don’t think they would imagine some contractor somewhere [...] is analyzing the conversations.”
Anonymous prompt worker, quoted by Joseph Cox, 404 Media reporter
OpenAI says it filters personal information first
OpenAI told 404 Media that it processes conversations through a version of its Privacy Filter model before sending them to contractors. The company says the system is designed to detect and remove personal information, but OpenAI’s own documentation acknowledges that the filter can miss uncommon identifiers, misunderstand ambiguous references, and over-redact or under-redact information when context is limited.
Reviewers do not see the user’s ChatGPT username. Some prompts can still contain identifying details, however, and the review dashboard may include a “user memories summary” describing earlier uses of the chatbot, including possible location information and other personal context.
Contractor instructions direct reviewers to escalate tasks involving potential safety concerns or personal information. 404 Media also found prompts suggesting that some users did not expect human scrutiny, including requests asking ChatGPT to keep the conversation private.
OpenAI did not answer 404 Media’s question about whether it explicitly tells users that humans may review prompts to improve responses. After publication, the company pointed to a help page stating that people may review content to improve model performance. OpenAI also updated its explanation of the setting that controls whether conversations are used to improve its models.
The default setting leaves many users opted in
OpenAI told 404 Media that users can prevent new conversations from being used to improve its models by turning off the “Improve the model for everyone” setting. The option is enabled by default for free, Plus, and Pro accounts, so users must change it themselves if they do not want new chats included.
The setting does not appear to apply retroactively, according to the report. Deleting conversations does not necessarily remove material that has already been de-identified and separated from a user’s account when the user has allowed OpenAI to use content for model improvement.
Enterprise, Business, and Edu customers have the model-improvement setting disabled by default. The distinction gives organizational accounts a different starting point from the consumer plans used by most individual ChatGPT customers.
Contractors rate style as well as accuracy
Project Lily is not limited to checking whether an answer is factually correct. Reviewers are asked to judge whether a response understands the user’s intent, offers helpful assistance, uses an appropriate tone, and avoids implying that ChatGPT has emotions or a personal history.
One instruction document tells reviewers to favor responses that are “helpful,” “honest & truthful,” “empowering,” and “smart, but humble.” Reviewers are not generally expected to fact-check answers through outside searches, although they must flag factual or correctness problems they notice. Separate teams apparently handle some verification work, and high-stakes medical, legal, and financial answers are expected to include sources when appropriate.
The documents do not identify which OpenAI model the contractors are training. The material reviewed by 404 Media uses only the Project Lily codename, leaving open whether the work supports a currently available model or a future release.
OpenAI is not the only company using human review
404 Media found that human evaluation is also part of the model-improvement process at other major chatbot companies. Google’s Gemini interface warns that humans review some saved chats to improve its AI systems. Anthropic told the publication that it uses human review for users who enable its “Help improve our AI models” setting and that it removes account identifiers such as email addresses before review.
Michal Luria, a senior research fellow at the Center for Democracy and Technology, told 404 Media that human review can support safety work but warned that chatbot interfaces create a misleading sense of privacy. Conversations can feel like private exchanges even when a company may route them to human evaluators.
Sarah T. Roberts, a UCLA professor and author of Behind the Screen: Content Moderation in the Shadows of Social Media, compared the arrangement to “the Wizard of Oz,” where the apparent magic depends on people working behind the curtain. The contractor interviewed by 404 Media said the work can be amusing but is generally repetitive, with shifting and sometimes contradictory guidelines.
The report puts a specific labor process behind the polished replies users see on ChatGPT: real conversations are selected, personal details are filtered imperfectly, and contractors score the system’s answers one by one. For consumer users, the immediate control is the model-improvement setting, which remains switched on by default for free, Plus, and Pro accounts.