Shiwen Dialogue 034 | Fifty Thousand Fake Citations Teach AI to Cite

【Column: Shiwen Dialogue】Host: “Shiwen” Guest: “Yongliang”

【Follow Guide】 👆 Click the blue text above “Yongliang Talks AI” to follow me, add a star mark, and get real insights on the workplace and business in the AI era, updated weekly.

【Cover Image: Title Card】

Opening Dialogue: When Fake Data Is Truly Cited

“Shiwen”: Recently, there was a popular post on Hacker News where a researcher named Metehan Yesilyurt spent fifty dollars to use GPT-4.1 to generate fifty thousand fake statistical pages in one day. Even more absurdly, these fake data did not pass Google’s filters but were continuously captured and cited by ChatGPT, leading to over ten thousand mentions over seven months. They even appeared in academic papers, investment research by Qatar Financial Group, Indian economics publications, and research reports by Canadian influencer communities. What do you think about this?

“Yongliang”: I read the content within the fact package. What hurts me most is not “someone faking”, but “faking being systematically taken seriously”. I have been in technology for seventeen years, managing teams of up to seventy people. In the past, when we discussed information quality, we always thought that as long as the search engine was good enough, the truth would always come first. But this case shows us that being cited by AI and being ranked by Google are two different things. ChatGPT can use a website almost deindexed by Google as an authoritative source for seven months. This is the most complex issue I want to discuss in today’s article.

“Shiwen”: So you think this problem is more profound than “AI nonsense”?

“Yongliang”: Yes. AI nonsense is a well-known problem, but here it’s not about AI creating content; it’s about AI extracting numbers from a seemingly formal web page, then layer by layer citing, packaging, and publishing these numbers, eventually turning them into “sources-based facts”. This is harder to prevent than simple hallucinations because it has the guise of a “citation chain”. Today, I want to break down this issue from four perspectives: phenomenon, personal experience, team mechanisms, and opposing boundaries.

Q1: What happened to these fifty thousand fake data?

“Shiwen”: Let’s first understand the phenomenon. How was this experiment conducted?

“Yongliang”: According to the information in the fact package, the author bought a new domain name, stateglobe.com, and used GPT-4.1 to generate approximately fifty thousand statistical pages within one day, covering SEO, content marketing, social media, e-commerce, internet technology, and two hundred countries. All numbers were fabricated; the author said, “I never checked a single one.”

“Shiwen”: What was the cost?

“Yongliang”: The total cost was about fifty dollars, with thirty dollars spent on generating statistical pages and twenty dollars on creating list articles. This cost is almost negligible in content marketing, but the output volume is astonishing. The key point is the subsequent development: after Google’s algorithm update, the site’s search visibility collapsed, and it was almost deindexed. In other words, it failed to pass the traditional search engine filter.

“Shiwen”: But ChatGPT passed the filter?

“Yongliang”: It’s gone, and it’s been well done. The facts in the package show that over the past thirty days, Cloudflare logs indicated more than six hundred thousand bot requests, of which about seventy-two thousand came from ChatGPT-User. Seven months later, this site is still referenced by ChatGPT, with GA4 showing over ten thousand sessions coming from ChatGPT. In Peec’s tracking of prompts, stateglobe.com ranks second in visibility, only behind Statista. This shows that ChatGPT’s retrieval and citation mechanisms do not consider “whether it’s indexed by Google” as a reliable filter. Not being ranked on Google is a different matter from whether AI can find and cite you.

“Shiwen”: Where have all these fake data gone?

“Yongliang”: The facts include four examples: an academic paper, investment research by Qatar’s largest financial group, an economic publication in India, and a research report from a Canadian influencer community. In other words, from academia to the financial industry, and then to self-media research reports, fake data has spread across various sectors.

Q2: How do you personally identify and deal with this “citation pollution”?

“Shiwen”: Speaking of your personal experience. With seventeen years as a technology director, how do you usually judge whether the data or citations provided by AI are reliable?

“Yongliang”: Let me start with a premise: This is my experience, not statistics. People’s trust in information often comes from “how formal it looks,” not “how much it has been verified.” A domain name, a table, or a citation format can easily make people relax their vigilance. So when I do fact-checking, I look at three things.

“Shiwen”: What are those three things?

“Yongliang”: First, check if there is a way to cross-verify the source. When AI gives me a number, I first ask: Where did this number originally come from? Is it from official statistics, academic journals, company financial reports, or some blog? If the source is untraceable, or if tracing back to a site I’ve never heard of, then I won’t accept it directly. Second, check if the data is internally consistent. Fake data is often isolated; it doesn’t match other data in the same field. For example, if a country’s e-commerce penetration rate is completely disconnected from its population, GDP, and internet penetration rate, there’s a question mark to be placed on it. Third, check if the citation chain has been “washed.” In this case, the author introduced the concept of citation laundering, where fake data becomes more and more like real data after being repeatedly cited. My experience is that the more “cited” something looks, the more I need to trace back to the earliest source.

“Shiwen”: So in this process, do AI tools help you or harm you?

“Yongliang”: A double-edged sword. AI can quickly help you organize information and generate summaries, but it also confuses “having a source” with “reliable sources.” I’ve encountered more than once where AI provided very standard citation formats, but when I clicked on them, either they wouldn’t open, or the content didn’t match. So I set a rule for myself: For any citation AI gives me, I either click to check it myself or cross-verify it with other sources. I won’t let AI complete the fact-checking alone.

“Shiwen”: Are there any scenarios where you definitely won’t let AI provide data directly?

“Yongliang”: Yes. There are three categories. The first category includes numbers related to investment, finance, and policy, such as market size, growth rates, and user bases. Once these numbers are incorrect, the cost of making decisions is very high. The second category involves reports, papers, and white papers published externally. Any content that will be signed or cited must have its data verified manually. The third category refers to scenarios where I need to make important judgments, such as whether to enter a market or invest in a certain direction. AI can help me quickly scan the materials, but the authenticity and interpretation of the numbers must be my responsibility.

Q3: How did you establish an information verification mechanism in a team of seventy people?

“Shiwen”: From individual to team. If you manage a team of seventy people, how do you prevent citation contamination caused by AI? “Yongliang”: Let’s start with a general principle: AI can generate content, but it cannot endorse the accuracy of the content. We have a strict requirement internally: all reports, PPTs, and public account articles published externally must indicate the original source, and this source must be traceable back to the first source. If AI was involved in generating the content, it must be stated in the document which parts used AI and which numbers were manually verified. This is not for formality, but to know where the responsibility lies when errors occur. “Shiwen”: How is this implemented specifically? “Yongliang”: We divided it into three levels. The first level is for internal discussion materials. Content generated by AI can be used directly, but everyone will mark “AI draft, not verified.” The second level is for internal decision-making materials. Data must have been verified by someone, and the names of people should be next to the core numbers. The third level is for external published materials. Every picture and every number must have its source stated, and it must be signed by the project leader. The boundaries of these three levels are clearly written in the team wiki. New employees should learn this first. “Shiwen”: But with so much information to handle daily, isn’t it unrealistic to verify everything manually? “Yongliang”: Of course not. That’s why we use mechanisms for layering. We did two things at that time. The first was to create a “data source whitelist.” Common data can only be obtained from official statistics, authoritative media, listed company financial reports, and well-known research institutions. AI can help find them, but whether to use them ultimately depends on the whitelist. The second was to establish a “sensitive number review list.” For key indicators such as market size, growth rates, user numbers, and revenue, there must be cross-verification from at least two independent sources, or confirmation by someone familiar with the field. “Shiwen”: Some colleagues think this adds burden. “Yongliang”: Definitely. Especially those in operations and marketing. They think I want things to be fast, but if you require me to verify every number, how can efficiency be maintained? My response is: AI aims for “finding quickly and writing quickly,” not “not being criticized when something goes out.” The cost of later refuting, clarifying, and restoring trust when an inaccurate number is cited is far higher than spending half an hour verifying once. I’ve proven this principle over many years of leading a team, so I dare to write it into the process.

Q4: How do opponents respond, and where are the boundaries?

“Shiwen”: There are also many discussions on HN. Let’s pick a few representative ones.

“Yongliang”: I’ll relay the comments from the fact pack. The first one comes from jqpabc123, which says: AI doesn’t care whether the data is fake or not. This statement is straightforward but points out a fact: AI’s citation mechanism is essentially probabilistic matching, not a judgment of truth or falsehood. It won’t refuse to cite a number just because it seems fabricated, as long as that number appears in a reasonable context.

“Shiwen”: What consequences does this judgment bring?

“Yongliang”: The second comment comes from zhisme, who explained more specifically: Some people still use ChatGPT to generate reports for managers, and the snowball of fake data will grow larger. In other words, once AI cites fake data, it will appear in more AI-generated reports, being cited by even more people, creating a self-reinforcing pollution chain. This is why the author calls it “citation laundering,” as the citation itself becomes a tool for washing things out.

“Shiwen”: Is there anyone who offers a solution?

“Yongliang”: Yes. The problem with ShadowOfThePit is: how can this be fixed? Nothing can stop you from posting nonsense online. This rhetorical question highlights the boundary: the nature of the internet is open publication, and no matter how advanced AI’s search mechanisms are, they cannot make all judgments of truth or falsehood for users. tkdb’s comment is also representative; he says that LLMs currently face the same problems as humans before Google: Are the information on this website true and high-quality?

“Shiwen”: So where is the boundary?

“Yongliang”: The boundary lies in “the user’s responsibility.” AI can reduce the cost of obtaining information, but it cannot replace you in taking responsibility for the consequences of your citations. If you are a researcher, analyst, or content creator, every minute you spend verifying the citations given by AI reduces your chances of being caught out later. If you are an ordinary reader, at least know that what AI cites may not have been verified, and key numbers should be checked back to their sources.

Conclusion: Fake data won’t disappear, but responsibility cannot be shifted

“Shiwen”: Let’s wrap up with a quote for today’s issue.

“Yongliang”: Fifty thousand pieces of fake data taught AI how to cite. The real lesson is not that “AI can deceive,” but that “AI is turning citation into a new form of pollution.” Before we said “with pictures comes the truth,” it became “with links comes the truth,” and now maybe even links and citations aren’t enough. The more powerful the tools, the more valuable human judgment becomes.

“Shiwen”: More specifically, what can ordinary professionals take away from this today?

“Yongliang”: Three sentences. First, the citations given by AI are not a way out of responsibility; no matter how long the citation chain is, ultimately you are responsible. Second, key numbers should be cross-verified at least once, especially those you write into reports, papers, or PPTs. Third, your team must have clear data verification mechanisms, with clear who claims, signs, and is responsible. No matter how technology changes, this will never change. Next time when AI helps you find data, don’t rush to copy and paste; ask yourself: If this data is questioned tomorrow, can I explain where it comes from?

【Follow Guide】 👇 Have you ever encountered AI citing fake data? How did you discover it? Share in the comments section, and we’ll pick some to discuss next time. Follow me for honest workplace insights in the AI era, updated weekly.

Information Source

• Peec AI blog post “Publish stats, get cited: The tactic that put my data all over the web,” published on October 7, 2026, by Metehan Yesilyurt (fact reproduction)
• Experimental details: stateglobe.com used GPT-4.1 to generate approximately 50,000 fake statistical pages within a day, covering 200 countries, at a cost of about $50 (fact reproduction)
• ChatGPT data collection: 30-day Cloudflare logs showed >600,000 bot requests, of which about 72,000 came from ChatGPT-User; after 7 months, GA4 showed over 10,000 sessions coming from ChatGPT (fact reproduction)
• Cited examples: academic papers, investment research by Qatar’s largest financial group, Indian economics publications, research reports by Canadian influencer communities (fact reproduction)
• HN comment pool: jqpabc123 “AI doesn’t know or care if the data is fake.”, zhisme “People still use ChatGPT to generate reports for their managers. A snowball of fake data,” ShadowOfThePit “How can this be fixed? Nothing prevents you from publishing straight BS…”, tkdb “LLMs are facing the same problem pre-google humans faced” (fact summary reproduction)
• Guest experiences: 17 years as technical director, peak team of seventy people; data source whitelists, sensitive numbers rechecking, AI content tagging, attribution confirmation, etc., all personal practices