The Pulse
MIT Study Finds GPT-5.5 Solves 34.6% of Theory Problems
An MIT FutureTech study found that OpenAI’s GPT-5.5 reached the same conclusions as human authors on 34.6% of sampled theory problems, up from 2.8% for o1. The researchers tested models on problems from 254 recent papers across seven scient
AI.info Team · Written with AI ·
OpenAI’s GPT-5.5 reached the same conclusion as human researchers on 34.6% of a set of recent theory problems, compared with 2.8% for the earlier o1 model, according to a study published October 13 by MIT FutureTech. The researchers tested models on problems drawn from 254 arXiv papers spanning seven scientific domains. Their reported comparison covers model releases from December 2024 to April 2026.
From o1 to GPT-5.5 in 16 months
The paper, by Gabriel Manso, Claudio Mayrink Verdun and Neil Thompson, measures whether a model’s answer matches the conclusion reached by the authors of each source paper. The study’s summary says: “Over these sixteen months, the share of problems for which models reached the same conclusion as the human authors rose from 2.8% (o1) to 34.6% (GPT-5.5), an increase of 22 percentage points per year.” Gabriel Manso, Claudio Mayrink Verdun and Neil Thompson, “AI Disruption and Falling Costs in Theory Research”
Researchers tested OpenAI models o1, o3, GPT-5, GPT-5.1, GPT-5.4 and GPT-5.5. The test problems came from research papers posted online between December 2025 and May 2026, according to WIRED’s account of the study. The researchers used problems whose published solutions postdated the models’ training cutoffs and blocked internet access during testing, limiting the chance that a model could retrieve an answer instead of deriving one.
A narrow test of scientific reasoning
The fields included probability and statistics theory, economic theory, high-energy theory, gravitation and cosmology, and quantum physics, alongside mathematics. The work deliberately focused on theoretical questions models could address without running experiments or simulations. That boundary matters: the results do not establish that models can independently conduct empirical research or reliably deliver correct answers across science.
Agreement with a paper’s human authors is the study’s scoring measure, not experimental confirmation of an answer. The researchers describe the human conclusions as ground truth, but the test still asks whether models reproduce those conclusions on a particular sample. The study’s overall rate therefore should not be read as a general success rate for scientific work.
About $15 per successful attempt
MIT FutureTech estimates that GPT-5.5’s runs cost about $5 per attempt, or roughly $15 per successful answer at the measured rate. The calculation makes the result relevant to researchers weighing whether AI assistance can lower the time and expense of theoretical work. It does not account for the expert effort needed to assess a solution or determine whether it is useful.
Thompson told WIRED that “Within just a few years, we expect a lot of theory questions to be totally transformed in terms of AI's ability to do them.” The researchers also caution that model outputs need scrutiny: Manso told the publication, “We are still going to need people to generate good ideas to connect the dots,” and said complex problems can expose mistakes. WIRED noted that OpenAI had withdrawn several solutions from a dataset after publication, a reminder that a plausible-looking derivation is not its own verification.
Experts remain part of the check
The study’s strongest claim is a measured jump in agreement with human-authored solutions on a defined set of recent theoretical problems—not proof that AI has replaced scientists. Its tested models all came from OpenAI, and its sample excludes research requiring experiments or simulations. For now, the paper puts a specific number on a narrower question: how often a model can reach the same conclusion as a researcher on a problem the model could not simply look up.