Thomas Bloom stops marking Erdős problems open or solved as AI-written proofs multiply

Bloom's decision came first. "Exponentially exploding," Bloom told Science News by phone, describing the flood of new results ^[14]. Soon after that call he posted that he would no longer tag problems as open or solved and closed comments, before he knew OpenAI would announce anything ^[4]. The 722 manuscripts landed hours after the call ^[1], on a site where visitors had already shifted from discussing problems to racing to close them ^[3].

His stated reason is workload: according to Science News, he does not want to spend his days refereeing AI-generated work ^[5]. Science News goes further in its own voice, arguing that competition among AI companies threatens to overwhelm human collaboration and understanding ^[18]. The support it gives is that those companies typically do not disclose their methods and data ^[17], plus one website's change in visitor behavior.

The DeepMind paper offers a denominator. AlphaProof Nexus solved 9 of the 353 open Erdős problems it tackled, reported October 8 in Science ^[6][7]. That is about 2.5 percent ^[15], and it measures one system on one list of open problems.

The sharper experiment is the control. The team ran a simpler setup of independent Gemini agents using Lean feedback, with no sharing of attempts and no AlphaProof, on the nine problems the full system had solved ^[8]. It solved all nine. It used fewer computing resources on the easy problems and more on the two toughest ^[8]. Chaudhuri was surprised, and Science News says the result suggests capable language models can tackle research mathematics with relatively simple checking tools ^[10].

What that design rules out is a need for the multi-agent machinery on these nine. It cannot rule out that the machinery matters elsewhere, because the simpler setup was only tried where the full system had already succeeded, and the team did not establish whether it would solve any problem the full system missed ^[9].

Lean is the check in both setups. It verifies only the statement it is given, as written in its code, and a statement can be mistranslated into or out of Lean ^[11]. An October 6 preprint found mismatches between parts of OpenAI's written Navier-Stokes proof and its Lean counterpart ^[12]. The authors write that this does not mean the written proof is wrong, but that a valid Lean proof does not automatically validate an AI-generated math paper ^[12].

DeepMind's report describes in detail how its results were obtained, and human mathematicians vetted many of the proofs ^[16]. Bloom said that is "quite different" from how OpenAI and Anthropic are handling their math results ^[16]. "The human mathematician has to really think through the problem and set goals for the system," Chaudhuri said ^[13].

In our view, the evidence supports a narrower claim than the threat Science News describes. The scarce step is checking. Lean confirms a statement, and a person has to confirm it is the statement the problem posed ^[11][13]. That holds for problems that can be formalized in Lean. Nobody yet knows how many of the 722 manuscripts are correct.