Lean proof checks cover about 300 of the 722 math manuscripts OpenAI's unreleased model produced

OpenAI decided what the field gets to grade. According to Crypto Briefing, the model was run against about 4,000 problems, and the catalog is the subset of outputs the company considered significant ^[10]. Set the 372 result families ^[1], or the more than 300 problems The Washington Post counted ^[19], against that 4,000 and the catalog covers between 7.5% and 9.3% of what was attempted ^[21]. Neither account reports how the model did on the rest.

Inside the catalog, checking splits in two. Lean is a proof assistant that checks every logical step of a proof the way a compiler checks code ^[9]. About 42% of the results, around 300, had Lean formalizations as of early October ^[8], and for those a machine has already confirmed that each step follows from the one before ^[9]. The other 420 or so manuscripts need a mathematician to read them ^[22]. Three retractions for errors shortly after release ^[11] come to about 0.4% of the catalog ^[23], counted in the first days of reading.

The Lean-checked results could hold and get cited and built on by human researchers, the test Crypto Briefing says mathematics ultimately applies ^[17]. Retractions could instead climb as readers reach the unformalized 420 ^[22]. Or part of the field could decline to read at all, and much of the catalog would sit unchecked.

I think review can settle the outputs and cannot settle the claim about the model. OpenAI has shared neither the underlying model nor the complete prompts ^[3], so a reader can confirm a proof but cannot rerun what produced it. The checking cost moves to mathematicians while OpenAI keeps the system. The counter-case is strong. A correct proof is correct whoever wrote it, and OpenAI says the work solved or made significant progress on hundreds of open problems ^[7]. The release also stops short of claiming any Millennium Prize Problem fully resolved ^[18]. If the unformalized results survive reading with few further retractions and OpenAI publishes more on its methods, the model question matters less than I am arguing.

September's Navier-Stokes announcement, reportedly about 10,000 AI agents working over 88 hours ^[2], or 880,000 agent-hours ^[20], drew criticism that OpenAI put speed ahead of independent verification ^[16]. This time the company added a GitHub repository with protocols for revisions and citations ^[12] and guidelines from an Institute for Advanced Study advisory group that drew on feedback from hundreds of mathematicians ^[4].

The Association for Human Mathematics says that group opened its first advisory statement by saying frontier AI companies should not test advanced mathematical problems on their internal models ^[5]. "We urge mathematicians to discontinue their work with OpenAI and to return to a vision of science that centres human understanding," AHM said ^[14]. Its 752 reported members could each take one manuscript and leave 30 people over ^[24]. They have pledged instead not to work with commercial AI companies or publish AI-generated mathematical writing ^[15].