The Pulse

Artificial Analysis reports VIALS-AA results for scientific image reasoning

Artificial Analysis’s VIALS-AA launch article reports results from testing language models on 161 scientific-image tasks. Gemini 4 Carbon (high) leads with a 64% score, while the evaluation shows differences in consistency, tool use and per

AI.info Team · Written with AI · fact-checked · 2026-11-07

VIALS-AA tests whether models can interpret scientific images

Artificial Analysis’s launch article presents VIALS-AA results as of November 6, 2026. Its implementation of Handshake AI’s VIALS benchmark tests whether language models can interpret scientific images and answer questions about the evidence they contain. Artificial Analysis runs each of the benchmark’s 161 tasks three times using Stirrup, its open-source reference harness. The harness gives models code execution and image-inspection tools.

Gemini 4 Carbon (high) leads with a 64% pass^3 score

Gemini 4 Carbon (high) leads with a score of 64%, answering all three repeats correctly on 103 of 161 tasks—two more than GPT-6 Astra (max). Artificial Analysis’s headline metric is pass^3, the share of tasks answered correctly in all three repeats. Kimi K3.1 Pro (max) and GPT-6 Astra (max) each solve around three-quarters of tasks at least once, while their pass^3 scores are 45% and 63%, respectively. The three leading open weights models solve about as many tasks at least once—114–119—as the seven top-scoring closed models, which solve 111–118; the open models pass all three repeats on 48–73 tasks, against 95–103 for the closed models.

More turns and visual inspection do not always get better results

The first task asks models to locate protein spot 22 in a gel electrophoresis image and estimate its molecular-weight range from the vertical axis. GPT-6 Astra (max) checks the original image and a 4× enlargement, then gives the correct 40–50 kDa answer in six turns without pixel measurements or contrast adjustments. Claude Sonnet 5.5 (max, with fallback) uses crops, pixel checks, reference lines and contrast enhancement before giving the same correct range in 18 turns. Grok 4.7 (xhigh) makes 27 turns and 20 image views, but answers 50–70 kDa; the reference answer is 40–50 kDa.

Artificial Analysis also tests GLM-5.3 (max), which lacks native vision, with the same image files and code-execution access. In the spot-22 trajectory, GLM-5.3 (max) takes 54 turns and 57 code calls, makes no image views and answers incorrectly. GLM-5.3-Flash (max) combines code with ten image views and finishes in 30 turns with the correct range.

Tool calls and per-task costs vary widely

The tool-use analysis covers 19 models and 9,177 rollouts. Image viewing accounts for 38.3% of tool calls, while preparing or measuring images—including creating crops, enhancing views and extracting measurements from pixels—accounts for 47.3%. Image measurement makes up 5–6% of tool calls for GPT-6 Astra (max) and GPT-6.1 Sol (max), compared with 25–27% for Claude Opus 5.5 (max) and Claude Sonnet 5.5 (max). Artificial Analysis reports that Gemini 3.8 Flash (high) accounts for 86.7% of the cohort’s commentary/no-op calls, which make up 5.6% of its tool calls, or about 3.8 of its 67.5 calls per task.

Domain leaders and per-task cost

Gemini 4 Carbon (high) leads or shares the lead in three of eight domain groups: plasmid maps, phylogenetics and flow cytometry. Claude Opus 5.5 (max) leads on blotting, Claude Sonnet 5.5 (max) on cell counting, GPT-6.1 Terra (max) on protein structure and binding, and GPT-6 Astra (max) shares the molecular-structure lead with GPT-6.1 Sol (max). Mean cost per task ranges from $0.02 for GPT-6 Luna (max) to $1.77 for Grok 4.7 (xhigh), a 99× spread. GPT-6.1 Sol (max) scores 59% at $0.06 per task, while Claude Sonnet 5.5 (max) scores 60% and costs about 18× as much. Artificial Analysis calculates task cost from token usage and model pricing, including representative cache-hit rates, and excludes judge calls.