The Pulse
Terminal-Bench-Science Puts Research Agents Through 70 Tasks
Terminal-Bench-Science 0.1 selects 70 research workflows from 920 proposals, with tasks graded by their own tests. An independent Artificial Analysis evaluation reports separate results using its own harness.
AI.info Team · Written with AI · fact-checked ·
Only 70 of 920 proposed tasks made it into Terminal-Bench-Science 0.1, the scientific-workflow benchmark released August 27. The project’s creators say 464 proposals passed an initial review for implementation, but domain and technical checks narrowed the first release to tasks they considered realistic, challenging and objectively testable. The result is a deliberately selective test of AI agents working through research tasks in a terminal.
“Terminal-Bench-Science evaluates AI agents on workflows from researchers' own work.”
Steven Dillmann, the announcement’s author
From 920 proposals to 70 tests
Researchers contributed tasks spanning life, physical, earth, mathematical and engineering sciences. The benchmark covers work such as data analysis, statistical inference, simulation, optimization, theorem proving and image reconstruction. Rather than quiz models with textbook questions, its creators aim to test workflows drawn from scientific practice.
Each task has its own tests, and an agent must pass every test for its attempt to count as a success. That standard measures whether a system produces the required result, not whether it can describe how a scientist might approach the problem. The release’s selectivity also reflects a practical constraint: tasks need to be scientifically meaningful and clearly specified enough for reproducible grading.
Independent results use a separate run
Artificial Analysis lists a separate evaluation of Terminal-Bench-Science 0.1. Its leaderboard gives GPT-6 Astra a 63.3% score, followed by two Claude Opus 5.5 configurations at 61.9% and 59.0%. Artificial Analysis says it runs the tasks with the mini-swe-agent harness and averages pass@1 scores across three repeats per task.
The benchmark’s own August 27 announcement reports a different set of results: Claude Opus 5 with Claude Code led at 30.0%, followed by GPT-5.6 Sol with Codex at 22.4%. These figures come from distinct evaluations; the project’s release post and Artificial Analysis describe separate runs, so their scores should not be read as a direct before-and-after comparison.
A lake’s freeze dates show the work involved
One example listed by Artificial Analysis asks an agent to estimate a lake’s freeze-up and ice-off dates across ten winters using bundled satellite images. The task supplies geospatial files and specifies thresholds for ice along a reference line and across the lake. The agent must work with the provided data and return dates that satisfy those criteria.
Such a task tests several linked steps—reading files, processing imagery and applying a scientific rule—rather than recalling a fact. Other tasks span different fields, but they share the benchmark’s emphasis on producing artifacts that can be checked against task-specific tests.
Scientists can shape the next release
The creators describe Terminal-Bench-Science as a continuous benchmark: future versions can add workflows, revise underspecified tasks and retire ones that agents have mastered. They say work on version 0.2 is underway, with a pull-request deadline of October 5, 2026. The project’s public contribution process asks scientists to propose tasks, implement them and submit them for domain and technical review.
The benchmark provides a way to measure agents on selected, verifiable research workflows; its pass rate does not establish how much of a scientist’s job an agent can perform. For now, the first release sets a concrete bar: 70 tasks across five fields, each graded against its own tests.