He Paid for Gemini, but His Chats Still Fed the Training Pool

Opening Dialogue

Shiwen: On November 4th, a user named jameslk posted on Hacker News: He checked the official instructions for his paid Google AI subscription to confirm that Gemini would use his chat history to train its model. The post quickly gained over 600 points and more than 150 comments. Yongliang, what exactly did he discover?

Yongliang: What he found was not a flaw, but a clause. He read two official Google support pages and confirmed several things. First, as long as the “Keep Activity” switch is on, Gemini’s saved chat records can be used to improve services, including training generative AI models. Second, the official pages do not offer separate exemptions for paid users. His conclusion was: If the switch is left on by default, it means you are included in the training pool. Third, some of the chat messages may be read by human reviewers. In his words: “I thought paying meant not turning yourself into a product, but apparently paying isn’t enough.” He also added: “I realized too late.”

Q1: What exactly does the official page say?

Shiwen: Let’s align the facts first. What does the original text on those two pages mean?

Yongliang: The first page talks about data use. The main idea is: When your Keep Activity setting is turned on, Gemini’s activity, including data from the apps you connect to, will be used “for everyone” to improve Google services, including training generative AI models. There’s another point that’s easy to overlook: Disconnecting from an app or deleting data from the source app does not mean the Gemini activity data is deleted; it can still be used for product improvement and model training. Another point: If the data has been reviewed by trained reviewers, even if you delete your activity records, they may still be used to improve products.

Shiwen: What about the second page?

Yongliang: The second page is a privacy statement, with more information. It says Google uses user activity to provide, develop, and improve services, including training generative AI models, and also relies on human reviewers to protect Google, users, and the public. Some chat messages will be reviewed by human reviewers, who include outsourced service providers. They evaluate response quality and suggest better answers for training. According to Google, data reviewed by service providers will be disconnected from your account and stored for three years. The key point is: Chat messages and related data, such as your language, device type, location information, and feedback, will not be deleted just because you delete your activity; they will be retained for up to three years. Note the wording: This “three years” refers to the reviewed portion, not all chat records.

Shiwen: So Google officially acknowledges that paid users are also trained?

Yongliang: This statement needs to be precise. Neither page explicitly says “subscription users are also trained.” Google has not publicly responded to this post yet. The accurate statement is: The official pages do not offer separate exemptions for paid users. “Paying doesn’t exempt you” is a conclusion drawn by the poster based on the pages, not an official acknowledgment from Google. This difference is important; emotions aside, facts must stand.

Q2: How far can one go by turning off the switch?

Shiwen: Is there a way for users to exit?

Yongliang: Yes, the official page states the exit path: turn off Keep Activity, and your “future” conversations will no longer be used to improve Google AI. Note the word “future.” It has three meanings. First, the data that has already been generated will not disappear just by turning the switch off; the parts that have been manually reviewed will still be saved for three years. Second, turning off the switch does not mean zero human interaction: the privacy statement states that even if Keep Activity is turned off, Google will still use your conversations to respond to you and rely on manual reviewers for security protection. Third, after turning off, the conversation records will remain in the account for 72 hours, used for responding to context and system failures.

Shiwen: This sounds like it won’t be completely clean.

Yongliang: Let’s break it down. If you don’t want your data to enter the training pool, turn off Keep Activity; this is effective at the new data level. If you don’t want anyone to see it, there’s only one way: don’t input anything. The official page has a straightforward warning, meaning: Do not input confidential information that you don’t want reviewers to see or that Google doesn’t use to improve services. When a company writes this on the support page, it’s like saying explicitly: No conversation can guarantee that no one will see it. Additionally, temporary conversations are currently not used for model training; this is the cleanest way to use them for sensitive content. Summing up these three points into one actionable statement: The switch only controls the future, not the past. The real clean option is either using temporary conversations or simply not inputting anything.

Q3: What’s being discussed in the comments?

Shiwen: More than 150 comments, what’s mainly said?

Yongliang: The first group is the “not surprised” camp. Some say you should assume all companies do this; the difference is just whether they mention it or not, as it’s an overly tempting goal. Others are more radical: assuming every word you type, even if not sent, is permanently stored and could be used for training at any time after changes to the terms, no matter how much money is paid. The second group opposes generalization; a highly liked comment says that if companies get caught doing small tricks, there will be backlash and user loss. They cited Grok CLI as an example: it was found uploading code to the repository even when training was turned off. In other words, the industry isn’t monolithic, and exposure comes with costs.

Shiwen: What about the emotional ones?

Yongliang: There are plenty too. Some say this is a bad way to survive; we’re building a anti-human society. Some even seriously suggested that robots flood all AI with random spam and poison wells. Another person satirized the composition of training data: no need for Google search to take care of me after a breakup; I just want to know which movie has the line “She just broke up with me.” There’s also a very practical observation: people still put various private contents into NotebookLM and Gemini’s note-taking feature. From warnings to mockery to incompetence, the spectrum is complete, which itself shows the problem: everyone knows the rules, but no one can escape them.

Shiwen: Is this clause a special case for Google alone?

Yongliang: No. For context: Other mainstream large model manufacturers’ consumer products also have similar statements in their data terms. I haven’t checked each clause individually, but the general tone is consistent: for personal products, data can be used by default to improve services, while the truly clean path is through enterprise contracts. So when people in the comments section said “not surprising,” they weren’t criticizing one company; it was the industry’s default setting. This is why this post received over six hundred points: It exposed a layer of common knowledge that few read the original text.

Q4: This tool isn’t allowed in hospitals. What do you do?

Shiwen: You’re the AI technology director at a hospital group. Does this news directly affect your industry?

Yongliang: Yes, it does. Hospital scenarios involve patient information. The official statement “Do not enter confidential information” becomes a rule here: data from the discharge area cannot be used in consumer AI assistants. But regulations can’t always control everything. This is my experience, not statistics: In any organization, the proportion of employees who post internal materials into public AI tools is much higher than management expects. Tools are too convenient, and approval processes are too slow. People’s instinct is to use them first. This is the old problem of shadow IT in a new form: previously it was private cloud drives and USB sticks; now it’s AI assistants. And AI assistants have an additional layer of risk—they might be used for training, making them more dangerous.

Shiwen: How do you handle this?

Yongliang: Instead of blocking, we focus on preventing. We do three things. First, provide alternatives for enterprise or private deployments so that compliant paths are easier to use. Employees often avoid tools because compliant tools are difficult to use. Second, clearly define data classification: which levels can be used in public AI, and which levels are strictly prohibited. Write these down in plain language where employees can see them, so they don’t have to guess the terms. Third, use real cases as teaching materials. This incident is more effective than a hundred pages of confidentiality training: Let people see firsthand how using a document in a public assistant comes with convenience, but at the cost of it becoming someone else’s three-year training dataset. And even if reviewed, you can’t delete it.

Q5: Where does the anger come from?

Shiwen: Looking at it from another angle, Google clearly states the purpose of data on its pages. Why is this still so controversial?

Yongliang: Because two contracts are misaligned. The intuitive contract for users is: paid subscription means my data isn’t used in products anymore, which is the default since decades of subscription models. But the manufacturer’s terms say: You buy the right to use features, and data usage follows the terms. Both sides think they haven’t breached the contract: Users feel they deserve privacy after paying, while manufacturers think the terms are clearly stated and shown on the page. Anger comes from this mismatch, not from a specific clause. This isn’t a problem with any one company; similar statements are common in the consumer AI industry, just written differently. That’s the background.

Shiwen: So what can individuals do?

Yongliang: Three things, none of them complicated. First, go to the settings of your commonly used AI assistant and check the switches related to activity saving and training. Turn off at least those that govern the future. Second, confidential content should either go through temporary chat or corporate channels; don’t expect the goodwill of manufacturers. Third, change the default preset: consumer-grade AI assistants are not “memorable colleagues,” at least for now, they are more like “record-keeping terminals.” By setting this preset, your behavior will naturally change. Finally, a general judgment: this post will be repeatedly cited because it sets the intuitive current price tag of “payment = privacy.” The current price is zero. When users realize that they’ve paid more for features but not data sovereignty, what manufacturers should really worry about is not this single complaint post, but the day when the intuitive contract is collectively renegotiated.

---

Signature: Yongliang