Hot Tracking 015 | Scolded for Three Months Before This Release: A Word-Deed Audit of OpenAI's 722 Math Manuscripts

First, let’s pin down the timeline. On October 6, OpenAI published the blog post ‘Sharing AI progress in mathematics’ and on the same day opened the repository openai/math on GitHub—gaining 9832 stars in two days, a speed that is extremely rare for academic repositories. The corresponding Hacker News discussion had accumulated 1241 upvotes and 1412 comments as of the morning of October 8, and is still brewing at the top of the charts. This is not a routine open-sourcing: from August to early October, OpenAI’s release method in the field of mathematics was continuously criticized for three months, and this major release occurred after the wave of criticism—the chronological order itself is what this issue aims to audit. This issue is also written in the middle of a continuously brewing event; the chart position is still moving, and the numbers are based on the snapshot of the morning of October 8. The following facts are based on the official blog and repository readme; HN reaction pool data has been independently verified, and cited parts are annotated in the main text.

What Happened

OpenAI publicly released in bulk the mathematical outputs of an internal model—722 research manuscripts, categorized into 372 result families, with the repository containing 132,880 files and 3.1GB. The primary language is the formal proof language Lean, under the Apache-2.0 license.

The repository is divided into three layers. The first layer is preprints, containing the PDFs plus source files of the 722 papers; the second layer is the lean directory, where 120,000 Lean files are organized by 24 disciplines, depending on a fixed version of the community math library mathlib4; the third layer is reasoning_traces, containing 10 excerpted summaries of reasoning processes—note that the official wording itself is ‘excerpted summaries’, not the complete thinking process. The verification entry point is also ready: the official recommendation is the comparator workflow, configuring the three tools comparator, landrun, and lean4export into the environment, pulling dependencies with lake update and fetching the community compilation cache with lake exe cache get, enabling verification of the result families one by one. The model that produced these results was not released; the official statement is ‘an unreleased internal frontier model’; it tackled about 4000 problems, spending an average of 3 hours of ChatGPT Pro-equivalent thinking compute per result. Out of the 722 papers, 162 have Lean-formalized main results, while the remaining 560 only have the papers themselves. The blog post also promised to fund a series of workshops and stated they are working towards ‘responsibly releasing the model that produced these results’—when the model itself will be released is a hook left for the future. The internal structure of the repository and how to run the verification process are specifically covered in the breakdown 019 published the same day, and will not be expanded upon here; the main focus of this article is something else: how this release was forced out by backlash.

Three Months of Backlash Before This Release

Let’s lay out the timeline first.

In August, OpenAI used the Astra model to demonstrate solving ten math problems in the Lean language—formalization tools are themselves public infrastructure for the math community, and an AI company using them as a showcase for model capabilities made some mathematicians feel uneasy for the first time. On September 10, a claimed proof related to the Navier-Stokes equations appeared—this is a century-old problem in fluid mechanics, and an unpeer-reviewed claim released directly to the public was predictably glaring. On September 11, 25 Fields Medalists published a joint statement directed at AI companies’ release practices in the field of mathematics; we wrote about this previously (Hotspot 003, September 12). In mid-September, mathematicians’ criticisms of the release practices continued to be reported, with the focus shifting from a single event to the release mechanism itself. Then came October 6: the blog stated that this release was designed according to the recommendations of the Mathematics and AI Advisory Group.

The most striking point of this attribution chain lies within the advisory group’s recommendation document itself. According to quotes from HN discussions, the recommendation document from this advisory group affiliated with the Institute for Advanced Study, Princeton (AGMAI) states two points right at the beginning: explicitly declining to endorse this practice, and requesting a stop to testing advanced mathematics problems on proprietary models. Comparing this with OpenAI’s actual behavior, the audit of their words and actions is clear: the release standards half—building a public repository, providing citation guidelines, doing Lean formalization, attaching reasoning summaries, and specifying compute parameters—was fully adopted; as for the “stop testing” half, the original blog dedicated entire paragraphs to arguing why it is important to continue testing math problems on internal frontier models. They listened on how to release; they didn’t listen on stopping testing. They took half from each side. This choice itself reveals their priorities: procedural justice is negotiable, but the pace of testing is not. The blog describes this release as a model of cooperation with the mathematics community, yet the hardest condition for that cooperation—stopping the testing—does not appear in any paragraph. As of press time, OpenAI has given no positive response to the demand to “stop testing.” Fields Medalist Terence Tao posted on Mastodon (cited via HN), with a more moderate attitude than the joint statement: he hopes AI can explain results to human mathematicians, and believes AI can make positive contributions in exposition, community building, and new research directions—but however moderate, he likewise did not say “please continue testing.”

A third layer of criticism is directed at the selling point of “machine verifiability.” 162 out of 722 yields a formalization rate of about 22%; the remaining 560 papers only have PDFs, with no Lean proofs—in other words, for nearly 80% of the manuscripts, the safeguard of machine verification is absent. Moreover, formalization itself is not a free pass: the paper arxiv 2610.08144 points out that the fidelity decision problem for automated formalization is SCI=∞ in the SCI hierarchy, making it even harder than the halting problem (SCI=1)—in plain terms, determining whether a machine translation has faithfully reproduced the original paper has even less of a general algorithm than determining whether a program will halt. That is to say, Lean verifies the post-formalization statements, but cannot guarantee that the formalization itself is a faithful translation of the original paper. Similar questioning on HN received high upvotes: the premise of a Lean certificate is that the theorem statement has been correctly translated, and across such a broad range of fields, there is no such guarantee. The repository README itself admits: the unformalized parts may contain errors, which will be fixed as soon as possible. The statements from all three parties are rarely this consistent—official, paper, and community all agree that a verification vacuum exists; it’s just that the official wrote it as a disclaimer paragraph.

What the Comments Say

With over 1,400 comments on HN, a few voices stand out as representative.

The most upvoted comment gave a positive rating to OpenAI’s interaction with the math community, but added a brutally sharp caveat: this was only done after being publicly called out. Another line of opinion goes even further: what the math community wants has never been permission, but norms—accusing the previous release process of scooping human researchers’ directions and failing to cite existing work (as reported in the discussion threads). Such accumulated grievances cannot be wiped away by a single massive release.

The release channel itself is also debated: some believe that an open repository with Lean code is a new paradigm for mathematical publishing, while others insist that papers should go on arXiv and undergo peer review; GitHub cannot replace peer review, and the presence of code cannot outweigh the problem of a missing review process. In mathematics, a field where peer review is the foundation of trust, the debate over channels is ultimately a debate over trust.

The most relatable comment came from a PhD student: what do you do when the direction of your half-written paper gets hit by 722 papers? The empathy below was very direct—don’t feed research to AI; it will be used for training and then to scoop your work. The attribution issue was also raised here: the 722 papers only credit OpenAI, not the specific researchers; someone suggested at least letting the reviewers be credited, and the response was that this equates to training the public to trust the machine rather than the reviewers; others explained that in a competitive environment, the team roster is inherently inconvenient to make public. Three different stances are squeezed into one attribution column.

There is also a widely circulated personal estimate: this batch of results is equivalent to 50 to 100 years of progress in mathematics—note that this is a single user’s estimate, not a consensus, and there are quite a few dissenting voices in the comment section. There is no need to hastily take sides on such assessments of scale: the repository is public, the results will be digested by the math community paper by paper, and time will tell their true worth.

Three Things Ordinary Users Can Do

First, verify a theorem by hand. The repository documentation provides the complete comparator workflow; by setting up the environment according to the steps in lean/README, a single command can verify a family of results—such as the 7/8 result of the quasi-Riemann Hypothesis. The best human record is 11/12; 7/8 is an advance weaker than the human record, not the endpoint. This is also the first time ordinary readers can do first-hand verification of cutting-edge AI mathematics claims—no need to understand the proof details, just run the command following the documentation. The README also specifically warns that compiling the entire library will hit the system memory mapping limit and provides bypass parameters—the repository is so large that it requires dedicated compilation troubleshooting documentation, which in itself is a footnote.

Second, use CONTENTS.md as a search map. The 722 articles should not be read through from the beginning like a magazine; the repository officially lists the result families in this file according to 24 disciplines. First, find the few entries of interest in your own field, and then enter the corresponding manuscript. Prioritizing search over reading through is the right approach when facing output of this magnitude, and the only way to save time.

Third, maintain a reading attitude of “unformalized = doubtful” towards conclusions that only have PDFs. The 560 manuscripts without Lean proofs should be read as proof claims, not verified facts—this is not being harsh, but the standard set by the repository’s README itself. Including this label when quoting, paraphrasing, or disseminating derivative works is the most practical thing ordinary readers can do for the information environment.

Conclusion

Ultimately, a proper sense of proportion was maintained. This issue records the chronological sequence and textual comparison; we do not speculate about OpenAI’s motives, nor do we pass judgment on it. It remained silent on the advisory group’s appeal to “stop testing”; the silence itself is a documented fact, not an answer we supply on its behalf. The statement that this release only came after criticism is a summary of the timeline, not a verdict on motives.

Two things are worth keeping an eye on next. First, will the model that produced the 722 results be released responsibly as the blog promised—whether it is released or not will directly test whether the sincerity of “how to release” stops at the manuscript or extends to the model itself. Second, when will the first piece among the 722 to be falsified or corrected appear—given the official premise that it was not formalized and may contain errors, this timing will largely determine how this large-scale release is ultimately evaluated: as an honest demonstration, or a batch of claims awaiting verification. How quickly they respond to the README’s promise to “fix as soon as possible” is also on our watchlist.

References

• OpenAI blog “Sharing AI progress in mathematics” (2026-10-06, Wayback full archive, primary source)
• github.com/openai/math (Primary source: README, CONTENTS.md, formalization.yaml, comparator docs; 722 manuscripts / 372 families / 132,880 files / 3.1GB / 9832★)
• Hacker News story 49984923 (1241▲ / 1412💬, Algolia data; views in the comment section are all paraphrased without quotes, the AGMAI proposal and Terence Tao’s post are cited from the discussion section)
• arxiv 2610.08144 “Navier–Stokes Lost in Translation” (abstract from primary source; source of the SCI=∞ conclusion)
• Related reports on the joint statement by 25 Fields Medalists (9-11; covered in this account’s hotspot 003, backreference)
• Astra 10 problems, Navier–Stokes claim, mid-September critical reports (secondary source pool including MSN archive, ForkLog, The Next Web, used for timeline anchors)