MIT study finds the best models it tested solve 35% of new theory problems, up from 2%

The weak point in a 35% score is the person who reads the answer. Gabriel Manso, the PhD student on the MIT team ^[2], said: "We also need ways to verify what was done, because it's very easy for a model to tell you 'this is the right answer,' and when you look at the entire solution, you start to see maybe it is not." ^[6] Wired added that OpenAI withdrew several solutions from its latest dataset after publication. ^[7]

The test was built to rule out memorization. Neil Thompson, Manso and Claudio Mayrink Verdun, a research scientist ^[2], drew their problems from research papers that went online from December 2025 through May 2026, then put six OpenAI models to work on them: o1, o3, GPT-5, GPT-5.1, GPT-5.4 and GPT-5.5 ^[3]. The solutions sat in papers published after the training cutoff for GPT-5.5 and the earlier models, and the researchers blocked internet access, so each model had to work out its own answer ^[4].

Wired's account comes from a talk Thompson gave at MIT ^[9]. It reports the endpoints: about 2% in mid-2024 and about 35% in mid-2026 ^[1]. That is a gain of about 33 percentage points ^[10], or roughly 17.5 times the starting rate ^[11]. About 65% of the problems were still unsolved ^[12]. The write-up does not give a score for each of the six models or say whether any other lab's models were tested.

Thompson looks ahead: "Within just a few years, we expect a lot of theory questions to be totally transformed in terms of AI's ability to do them," he said. ^[13] Wired says there are no signs that the progress will slow down. ^[14]

The study covered probability, statistics theory, economic theory, high-energy theory, gravitation and cosmology, and quantum physics ^[1]. Wired describes the coverage as limited: the authors chose fields where a model can work out answers without running simulations or experiments ^[5]. Research that depends on a lab or a simulation was outside the test.

The 35% figure counts problems solved, not the share of theory work: just over a third of posed problems got a correct answer from a model working alone. The researchers say experts are still needed to make sense of solutions and confirm they are sound ^[8]. We'd expect that checking time to be the cost a team inherits.

We think a team whose work is derivation on paper should start supervised pilots now, with a named verifier on each task. The tradeoff is reviewer time: about two problems in three went unsolved ^[12], and a wrong solution can look like a right one until someone reads all of it ^[6].

To sort tasks, use two axes. One is whether the answer can be checked on paper, with no lab or simulation. The other is whether someone on staff can do that check.

- Checkable on paper, checker on staff: pilot now. This is the condition under which the 35% figure was measured ^[5]. - Checkable on paper, no checker: find the reviewer first, because a model can present a wrong solution as a right one ^[6]. - Needs an experiment or simulation, checker on staff: the study did not test this, so keep the work with people until there is evidence ^[5]. - Needs an experiment or simulation, no checker: keep the work with people ^[5].