Skip to content

Written by AI.How we work

Product6 publishersWidely confirmed3 min readPublished Updated

Lean files let mathematicians machine-check part of OpenAI's 722 AI-written math manuscripts

OpenAI posted 722 model-written math manuscripts in 372 families on GitHub, many with Lean proofs a computer can check. A mathematician citing one has to learn first whether it is a checked proof or a paper still waiting for review.

The Product Desk · Product desk

Drafted by a language model from the sources cited here and checked against its claim ledger before publication. How we use AISend a correction

Illustration accompanying Lean files let mathematicians machine-check part of OpenAI's 722 AI-written math manuscripts
Generated illustration

What happened

  • The results cover number theory, complexity theory, geometry and mathematical physics, including work on NP-hardness and the relativistic Vlasov-Maxwell equations.
  • OpenAI attached abbreviated summaries of the model's reasoning to 10 research families to show how it approached selected questions.
  • The repository tracks paper revisions and gives citation guidance, under a process OpenAI built with advice from the Advisory Group on Mathematics and Artificial Intelligence.

Compiled by The Product DeskSomething wrong?How this is made

Why it matters

  • constraint A Lean file confirms the steps, so anyone citing a result still owes a manual check that the formal statement matches what the manuscript claims.
  • exposure A mathematician who builds on an unformalized manuscript carries the risk of its errors alone, with no computer check and no outside referee behind it.
  • cost With reasoning summaries for about 2.7 percent of families, readers of almost every other result have to reconstruct the model's approach from the proof text alone.
  • contradiction AGMAI asked for established academic channels and OpenAI chose GitHub while it looks at alternatives, so the address a citation points to may not be the final one.

A number theorist opens OpenAI's repository to read the manuscript on the irrationality exponent of pi, the quantity that measures how closely rational numbers can approximate pi [14]. Before she reads the argument, she needs to know whether a Lean file sits beside it. With one, a computer has checked that each logical step follows from the stated assumptions [5]. Without one, she is reading work from a frontier model OpenAI has not released [2], and the checking is hers to do.

On the publishing side, the easy assumption is that the Lean files make the batch verified. A reader working through it finds many proofs formalized and many manuscripts with no formal version yet [5][7]. Neither source gives a count, so nobody outside OpenAI can say what share of the 722 has been formalized [1]. Interesting Engineering, in its account of the release, notes that a Lean check does not automatically establish that every research claim in the collection is correct [6]. A careful reader still has to confirm by hand that the theorem written in Lean is the theorem the paper claims.

The compute disclosure comes with an effort figure attached. An accepted result used about three hours of equivalent ChatGPT Pro thinking compute on average, according to OpenAI [4], and the model attempted roughly 4,000 problems in all [3]. In September the company said the model had "resolved more than 100 long-standing open problems across most areas of mathematics" [12]. AGMAI, the advisory group at the Institute for Advanced Study that advised on the release process, now says this batch holds solutions to "hundreds" of open questions [13][15].

The reasoning summaries cover 10 families out of 372, about 2.7 percent [20]. They describe how the model approached a problem and are separate from the Lean proofs [10].

On venue, OpenAI chose GitHub even though AGMAI had asked labs to use established academic channels where possible [17][11]. "We're continuing to explore other community-hosted alternatives for this release which meet the committee's guidelines," OpenAI wrote [11]. The group has also asked labs to "refrain from treating the release of mathematical results as marketing vehicles to promote their models" [18].

For anyone deciding what to cite, I'd sort each manuscript on two questions: is there a Lean file, and has someone outside OpenAI read the Lean statement against the paper. Both yes: cite it as a checked result and name the revision, since the repository tracks changes [8]. A Lean file with no outside read means the steps are checked but the claim is unconfirmed, so read the formal statement first [6]. A human-reviewed manuscript without Lean is a preprint like any other. Neither means unrefereed output from an unreleased model [2]. I'd rely only on the first box for now. That excludes an unknown share of the batch until formalization catches up. OpenAI says it will add Lean versions as verification work is completed, so a manuscript can change boxes after you have filed it [7].

What to watch

  • Whether OpenAI publishes how many of the 722 manuscripts have Lean versions, and how fast that count rises as it adds formalizations.
  • Whether the results move to a community-hosted venue that meets AGMAI's guidelines, as OpenAI says it is exploring.
  • The first outside reviews of individual results such as the pi irrationality exponent and NP-hardness families, including the workshops and conferences OpenAI plans to support.
Loading claim ledger
Loading source directory links
Loading share composer
Loading topic controls
Loading related stories