The Pulse
FrontierPhysics Tests Agents on 56 Physics Research Tasks
FrontierPhysics uses 56 tasks drawn from physicists’ own research to assess whether AI agents can complete research across nine physics domains. In tests spanning 22 models and 26 configurations, no configuration passed more than 46 of its
AI.info Team · Written with AI · fact-checked ·
A benchmark for end-to-end physics research
A preprint submitted to arXiv on 4 November 2026 introduces FrontierPhysics, a benchmark designed to test whether AI agents can carry out research across theoretical, computational and experimental physics. Its 56 tasks were created by 42 physicists from their own research and span nine domains.
The paper argues that many existing physics benchmarks cover fixed questions, isolated subtasks or reproductions of papers for which the solution is already available. Such tests, the authors write, rarely assess research quality alongside correctness. FrontierPhysics is intended to evaluate a wider process, rather than only an answer to a defined question or performance on one part of a research workflow.
Tasks draw on researchers’ own work
The task authors are active physics researchers, including Ph.D. students, postdoctoral researchers and research scientists. According to the preprint, each task is based on a research project its author personally conducted. The authors report an average of 420 citations per task author.
Tasks go through a general review and a domain-expert peer review, followed by failure-mode analysis. The paper says this process is intended to retain open-endedness while reducing false positives and false negatives. It also reflects the authors’ view that task design and assessment require specialist judgment: a model’s failure may reflect a lack of expert intuition or experience, rather than a basic misunderstanding of physics.
Agents must plan, investigate and produce research outputs
Each task gives an agent a high-level prompt resembling instructions a principal investigator might provide at the start of a project. Agents must review relevant literature, develop a research plan, conduct research and experiments, and produce structured outputs and scientific manuscripts.
FrontierPhysics combines deterministic verifiers with expert-authored rubrics to assess whether the work is correct and publishable. The paper’s overview describes a deterministic verifier and a reviewer agent using the task rubric to grade each rollout. It also says the benchmark’s administrative team and task contributors manually examined failure modes in 168 rollouts from the GPT-6 Astra configuration.
The preprint places this approach against earlier research benchmarks that test scientific reasoning or individual capabilities within realistic workflows. It also discusses full-paper reproduction tasks, where the original publication can give agents the research question and solution structure in advance. Other benchmarks turn published studies into research problems without revealing solutions, but the authors say building and assessing those tasks still involves decisions that benefit from the original researchers’ expertise.
Reported results across models and configurations
The evaluation covered 22 models in 26 configurations. The authors report that no configuration passed more than 46 of its 168 rollouts. The paper’s overview also describes 4,364 rollouts across 22 models and four harnesses, with each rollout representing one trial of a task by a model in an agent harness.
“Manual review of agent trajectories shows that frontier agents’ main weaknesses are insufficiently rigorous reasoning and limited expert intuition, rather than gaps in physics knowledge or execution errors,” the authors write. The finding distinguishes the weaknesses identified in their review from shortcomings in physics knowledge or execution.
The preprint is listed on arXiv under Artificial Intelligence. Its arXiv record gives the submission date as 4 November 2026 and describes the paper as 41 pages, including 10 pages of main text, with 19 figures and 11 tables.