ChatGPT Conversations Under Human Review: OpenAI's 'Project Lily' Leak

A striking development has emerged that is set to reignite debates over privacy and data security in the AI ecosystem. According to internal company documents and employee statements obtained by 404 Media, OpenAI has human employees and contractors review the conversations of real ChatGPT users as part of a secret internal project.

Code-named "Project Lily", this extensive operation involves hundreds of staff reading user dialogues line by line and grading the answers the AI produced. But the part of the work that has caused the biggest stir is that users' private conversations and sensitive data can come into contact with human eyes in the process.

What Is Project Lily and What Is It For?

The leaked documents show that Project Lily was designed to improve the AI's answer quality, accuracy, tone and reasoning ability. Reviewers grade the outputs ChatGPT produces for user questions according to predefined parameters.

One of the project's most critical focus areas is reducing the behavior known in the AI literature as "sycophancy" (a tendency toward flattery or excessive agreement). Language models often try to please the user by endorsing flawed arguments, giving in to the user's wrong steering, or offering needlessly glowing answers. With human feedback, OpenAI aims to make the model take a more principled, neutral and critical stance.

Personal Data That Slips Past Anonymization

The most sensitive aspect of the report, and the one causing public concern, relates to data privacy violations. Although OpenAI reportedly removes user names and tries to scrub personally identifiable information (PII) through its system, the filters are said not to work flawlessly in every scenario.

Reportedly, contractor reviewers can at times witness situations like these:

• Incomplete masking: Addresses, internal company strategies, personal names or private contact details embedded in the body of the text may not be fully filtered out.
• The entire conversation tree: Reviewers see not just a single prompt-answer pair, but the entire deep and extensive conversation chain between the user and ChatGPT.
• Contextual clues: Even when a name has been removed, the flow of the conversation and the private details shared can make it possible to infer the user's identity or the organization they work for.

Most users assume that the sessions they have with ChatGPT at their screens are shared only with an algorithm. The possibility that a data labeling contractor thousands of kilometers away could read those dialogues shakes trust in the platform.

The Contradiction of RLHF and Human Evaluation

Huge datasets collected from the internet are not enough on their own to develop large language models. Reinforcement Learning from Human Feedback (RLHF) is considered a necessity for aligning models with human values (alignment) and giving them logical consistency.

However, feeding the raw conversations of the live user base directly into this machine, instead of synthetic data or volunteer subjects, brings ethical and legal debates with it. People are questioning to what extent the vague "data may be used to improve the service" clauses in terms of use agreements legitimize having conversations read by third parties.

How Can You Prevent Your Conversations From Being Reviewed?

If you don't want your conversations with ChatGPT to be used in training OpenAI models and in human review processes, it is very important to actively configure the privacy settings the platform offers:

1. In the ChatGPT interface, click your profile icon at the bottom left and open the Settings menu. 2. Switch to the Data Controls tab. 3. Turn off the "Improve the model for everyone" option, or the chat history training option. 4. For corporate or sensitive projects, prefer working through the API; OpenAI's official API policies commit that user data is not used for model training by default.

Also, as a basic security principle, you should never paste critical information such as passwords, proprietary source code, health records and legal secrets directly into any public AI interface.

A Statement From OpenAI Is Awaited

For now, these findings, which have taken center stage, are based on documents obtained by 404 Media, interviews with internal sources and internal company communication records. It remains to be seen whether OpenAI's leadership will make an official statement that satisfies the public about the scope of the reviews, which account types are affected and how the data is isolated.