Technical Deep Dives

The Benchmark Problem: How We Measure AI Intelligence — and Why It Keeps Breaking

Every AI benchmark saturates, gets gamed, or breaks in public. From GLUE and MMLU to ARC-AGI, FrontierMath and Humanity's Last Exam, here is what happened, with dates and numbers — ending on one model scoring 62.7% or 99.9% on the same test

Gabriele Masetti · 2026-09-10

The Lifecycle Nobody Designed For

Every AI benchmark now follows the same arc, whether its authors intend it or not. A team proposes a test that current models fail badly. Labs chase the number. Scores climb until the test stops separating good models from great ones. Someone builds a harder replacement, and the cycle restarts. That sequence has played out at least four times in eight years — GLUE to SuperGLUE to MMLU to GPQA/ARC-AGI/FrontierMath/Humanity's Last Exam — and the interval between proposal and saturation keeps shrinking.

None of this is a story about any one test being poorly designed; it is what happens when frontier capability improves faster than the yardstick can be redrawn.

The consequence is that "state of the art on benchmark X" has a shelf life, and increasingly the shelf life is public knowledge. GPQA's creators built in expert-vs-non-expert calibration from day one, expecting saturation. ARC-AGI's creator built an entire prize foundation on the assumption that any static test eventually gets gamed or genuinely solved, and pre-committed to a sequel before the first version finished its run. Treating benchmarks as durable ground truth, rather than instruments with a service life, is the root error behind most of the problems that follow.

GLUE to MMLU: A Decade of Rising Ceilings

GLUE launched in 2018 to test sentence-level language understanding across nine tasks. GPT scored 72.8 and BERT 80.2 on it that year. By early July 2019, Yang et al.'s XLNet had pushed the leaderboard to 88.4, edging past the human baseline of 87.1. SuperGLUE arrived in May 2019 because GLUE had become too easy — at launch it still showed roughly a 20-point gap between the best model and human performance, which was the point.

It didn't last: T5 reached 88.9, within a point of the human score, and by early 2020 Microsoft's DeBERTa and Google's T5+Meena had both crossed the human baseline of 89.8.

MMLU, released September 7, 2020 by Dan Hendrycks and coauthors (published at ICLR 2021), was the next reset. It covers 57 subjects — from elementary math to law to virology — across 15,908 multiple-choice questions. When it launched, most models scored near the 25% random-chance floor; GPT-3 175B, the best available, hit 43.9%.

By 2026 every frontier model exceeds roughly 88% on MMLU, and multiple trackers describe it as functionally saturated — a "floor check" useful mainly for confirming a model isn't broken, not for ranking frontier systems.

GSM8K (8,500 grade-school math word problems, 2021) and HumanEval (164 hand-written coding problems, introduced alongside Codex) both went from meaningful discriminators to near-ceiling exercises within a few years, which is why math and code evaluation moved on to GPQA, FrontierMath and SWE-bench.

The Leak Nobody Can Fully Rule Out

Saturation has an uglier cousin: contamination, where test questions (or close paraphrases) end up inside a model's training data, inflating scores without reflecting real capability gains. The problem is old enough to appear in the founding papers of the field. Brown et al.'s original GPT-3 paper (2020) disclosed in its own appendix that a bug in its filtering pipeline let benchmark data leak into training, and separately found that C4, a common pretraining corpus, already contained the test splits of several benchmarks scraped wholesale from GitHub.

OpenAI's GPT-4 technical report went further, describing how the team mixed portions of the MATH and GSM8K training sets into GPT-4's own training data to improve math performance, while running a substring-overlap check to verify the held-out test sets weren't duplicated.

Independent audits have kept finding it since: measurable contamination for Mistral-7B-v0.1 on HellaSwag and GSM8K, for Llama-3-70B on ARC, and for the Phi and Mistral families on GSM8K, where performance on a freshly written GSM8K-style set (GSM1K) trailed the original by as much as 13 percentage points — a gap consistent with memorization rather than reasoning. None of this means any single lab cheats on purpose; contamination can enter through web-scraped pretraining data years before anyone runs an eval. It does mean a benchmark score without a contamination audit attached is now treated, by careful practitioners, as a claim rather than a fact.

Building Benchmarks That Fight Back

The response has been to design tests that resist both saturation and contamination at once. GPQA, published by David Rein and coauthors (NYU, with Anthropic collaborators) on November 20, 2023, is 448 multiple-choice questions written by PhD-level experts in biology, physics, and chemistry, built to be "Google-proof." Its hardest subset, GPQA Diamond (198 questions), keeps only items where both expert annotators agreed and most skilled non-experts — given unrestricted web access and over 30 minutes per question — still got it wrong. Domain PhDs reach about 65% accuracy; GPT-4, at launch, managed 39%.

SWE-bench, and the OpenAI-curated SWE-bench Verified subset released August 13, 2024, took a different tack: instead of writing new questions, it pulls real, unresolved GitHub issues and asks a model to generate a patch that passes the project's actual test suite. Verified's 500 tasks were filtered by 93 professional Python developers screening 1,699 candidate samples for solvability.

It worked as a discriminator for barely a year — state of the art rose from 74.9% to 80.9% in about six months, by OpenAI's own tracking — before OpenAI published an audit in 2025 showing that of 138 problems its o3 model failed to solve consistently across 64 runs, 59.4% turned out to have broken test design.

OpenAI then stopped reporting SWE-bench Verified scores and pointed the field toward SWE-bench Pro — a rare case of a benchmark's own sponsor retiring it in public.

FrontierMath, launched by Epoch AI in November 2024, set the bar so high that all six models evaluated at launch solved fewer than 2% of its research-level math problems. Less than two years later, top scores had climbed past 87–88%, a more than fortyfold jump — fast enough that Epoch revised the benchmark on June 12, 2026 after finding errors in 42% of the original problem set, trimming it to 338 problems. Epoch has since grown the family rather than only pruning it: FrontierMath now runs Tiers 1 to 4 alongside an Open Problems collection of unsolved research questions with computationally verifiable solutions, and FrontierMath Erdős, open problems posed or studied by Paul Erdős, formalised in Lean.

ARC-AGI and the Prize That Won't Pay Out

François Chollet's ARC-AGI benchmark takes a different angle: abstract grid puzzles, easy for untrained humans and historically brutal for neural networks, designed to probe fluid reasoning rather than memorized knowledge. Chollet co-founded the ARC Prize Foundation with Mike Knoop to run it as an annual competition with prize money withheld until a real threshold is cleared.

ARC Prize 2024 produced the largest single-year jump the benchmark had seen since 2020 — state of the art moved from 33% to 55.5% on the original ARC-AGI-1 set — which is why ARC-AGI-2 replaced it in March 2025, built to resist the program-synthesis and brute-force search tricks that had cracked version one.

ARC Prize reports that every task in ARC-AGI-2 was solved by at least two of the 400-plus people it tested in San Diego in early 2025. ARC Prize 2025 drew 1,455 teams and 15,154 Kaggle submissions; the winning NVARC team took first with 24.03%, using a test-time-trained model built on Tiny Recursive Model components — well short of the prize bar even as a competition winner.

ARC Prize 2026 restructured the money rather than the threshold. The ARC-AGI-2 track carries $700,000: $275,000 in progress prizes, $275,000 for the best open-sourced write-up, and a $150,000 bonus for the first eligible entry to clear 85% on the private evaluation set, which rolls forward to 2027 if nobody does. A separate ARC-AGI-3 track carries $850,000, of which $700,000 is reserved for the first agent to score 100%.

Frontier labs reporting on the separate, uncapped-cost leaderboards moved far faster. The best published ARC-AGI-2 figures when this piece first ran were Claude Opus 4.5 with extended thinking at 37.6% for roughly $2.20 a task and Google's Gemini 3 Pro with an iterative refinement wrapper at 54% for about $30. By mid-September 2026 the public trackers LLM-Stats and BenchLM both put OpenAI's GPT-6 Astra at 95% and GPT-5.5 at 85%; BenchLM also has Claude Opus 5 at 90.4%.

A benchmark released in March 2025 to resist the methods that had cracked its predecessor was, on those numbers, finished inside eighteen months. The gap between a capped, open-sourced competition entry at 24% and an unconstrained frontier model at 95% is itself the finding: raw capability and prize-worthy capability are two quantities now, and only one is independently audited.

Humanity's Last Exam and the Limits of Making Questions Hard

Humanity's Last Exam, built jointly by the Center for AI Safety and Scale AI under CAIS director Dan Hendrycks — reportedly after Elon Musk remarked that MMLU had gotten too easy — launched in January 2025 with 2,500 graduate-level questions spanning math (41% of the set), the physical and life sciences, humanities, computer science and engineering, about 14% multimodal. It was formally published in Nature in January 2026. At launch, GPT-4o scored 2.7%, Claude 3.5 Sonnet scored 4.1%, and OpenAI's o1 scored 8%, against a rough 90% baseline for human subject-matter experts.

The exam's difficulty came with a cost the creators hadn't priced in. Because questions were selected specifically for being ones frontier models got wrong, and reviewers spent no more than five minutes verifying each rationale, quality control was thin. In July 2025, the research lab FutureHouse audited 321 text-only biology and chemistry questions with a literature-search agent and human review, and found roughly 29% (± 3.7%, 95% CI) had answers directly conflicting with peer-reviewed literature.

The HLE team's own follow-up found about 18% of a subset problematic. FutureHouse released a cleaned "HLE Bio/Chem Gold" subset, and the HLE team introduced "HLE-Rolling," a continuously revised version meant to absorb reported errors rather than freeze a flawed set in place.

Scores have moved a long way since launch regardless. Artificial Analysis's July 2026 tracking put the leading model at 53.3%; two months later LLM-Stats and BenchLM both put the leader at 65%, with the next three models inside half a percentage point of it. A test whose launch scores were 2.7% and 4.1% now separates the top of the field by rounding error, while still containing a meaningful share of wrong answers by its own team's audit. Making a test harder does not automatically make it more correct.

When Humans Grade the Answers: LMArena's Bad Year

Static benchmarks aren't the only kind under strain. LMArena (formerly LMSYS Chatbot Arena) ranks models by crowdsourced pairwise human votes, converted into an Elo-style score, and it hit its own credibility crisis in April 2025. Meta submitted a model to the Arena labeled "Llama-4-Maverick-03-26-Experimental" — a chat-tuned variant, not the model it was about to publicly release — and it landed at #2 on the leaderboard, just behind Gemini 2.5 Pro.

When the actual public release, Llama-4-Maverick-17B-128E-Instruct, was tested separately on April 11, it placed 32nd, thirty spots below its experimental stand-in. Researchers comparing outputs found the experimental version was notably more verbose and emoji-heavy, a style that appears to have won human preference votes independent of underlying quality; LMArena acknowledged that "style and model response tone" had given the tuned version an edge the disclosure hadn't made clear.

Weeks later, a team spanning Cohere Labs, AI2, Princeton, Stanford, Waterloo, and the University of Washington published "The Leaderboard Illusion," a 68-page analysis of roughly two million Arena battles across 243 models and 42 providers. It found that a handful of large proprietary labs — the paper names Meta, Google, and OpenAI — had access to undisclosed private pre-release testing, selective score retraction, and disproportionately high sampling rates relative to open-weight competitors, and that selectively submitting only the best-performing private variants had inflated some scores by roughly 100 Elo points.

LMArena published a response disputing parts of the framing. The lesson from both episodes matches what contamination teaches on the static side: a leaderboard is only as trustworthy as the submission process feeding it, regardless of how many humans vote.

Beyond the Score: Time Horizons and What Comes Next

The most useful reframing of the last two years didn't propose a harder quiz at all. METR, the AI evaluation nonprofit, published "Measuring AI Ability to Complete Long Tasks" in March 2025, defining a model's "50% time horizon" as the length of task — measured in how long it takes a skilled human professional — that the model can complete autonomously with 50% reliability, using a battery of software and research tasks ranging from one second to sixteen hours of human-equivalent effort.

Tracking frontier models back to 2019, METR found this horizon doubling roughly every seven months; in follow-up work through 2025, that doubling time compressed further, to roughly every four months. It reframes the question from "what score did the model get" to "how long a task can I hand this system and walk away," closer to what deploying AI in a real workflow actually requires.

LiveBench, a 2025 ICLR Spotlight paper, replaces roughly a sixth of its questions every month — sourced from fresh arXiv papers, news, and other post-cutoff material — so the full set turns over every six months and can't be memorized in advance; it also grades every answer against an objective ground truth rather than an LLM judge.

Private held-out test sets, domain-specific evals run inside individual companies, and live, rolling benchmarks are converging on the principle GPQA and ARC-AGI already assumed: a test that can't be gamed once and reused forever beats one that produces a clean, stable, comparable number.

Six Months of ARC-AGI-3, and the Number That Depends on the Harness

ARC-AGI-3, the interactive, game-style agentic benchmark Chollet launched on March 25, 2026, ran that experiment faster than anyone expected. Frontier models managed 0.51% at launch, against environments humans solve completely. On May 1, 2026 ARC Prize reported GPT-5.5 at 0.43% and Claude Opus 4.7 at 0.18% on the semi-private set. Six weeks in, the wall was still a wall.

On September 3, 2026 ARC Prize published its evaluation of OpenAI's GPT-6 Astra, and the wall was gone. Astra scored 62.7% on ARC-AGI-3 Semi-Private at a cost of $26,000. Run through what ARC Prize calls a Provider Adapter harness, the same model on the same tasks scored 99.9% for $19,000. ARC Prize also found Astra used fewer actions than the human baseline on 96.0% of levels.

Read the two scores together and the argument of this essay sits in the gap between them: one model, one benchmark, one evaluator, one publication, and 37 percentage points that depend entirely on the scaffolding the model is run through. ARC Prize was careful, writing that "while we believe Astra represents meaningful progress towards generalization, we are not claiming that it is AGI."

Submissions for ARC Prize 2026 are due November 2 and results are announced December 4; as of mid-September the $150,000 bonus for clearing 85% on the ARC-AGI-2 private set was unclaimed. What separates a 95% on a tracker from that unclaimed prize is not capability. It is internet access, open-sourcing, cost caps and an audit — the conditions under which a number means anything. The field spent eight years building harder tests. The harder problem turned out to be agreeing on how to run them.