Agent Plasticity: Measuring Self-Improvement Through Experience

AI summary

Overview

Research area: AI agent evaluation / continual learning — specifically, how well language-model agents improve themselves through experience without changing their weights.

Technical level: Intermediate. The protocol is conceptually simple (play games, write notes and tools, get re-scored), but the paper assumes familiarity with agent harnesses, held-out evaluation, and curve fitting.

Scope: The paper proposes a controlled protocol and a new metric, "agent plasticity," to measure whether, how efficiently, and where frontier agents convert experience into persistent gains on held-out tasks across Chess, Go, Hex, and NetHack.

What This Paper Is About

Standard agent benchmarks measure what a model can do at a single moment, which hides whether it is actually getting better over time. The authors ask three questions: does performance improve and generalize beyond the interactions that produced the learning, how efficiently is new capability acquired, and where does the improvement loop break down? To answer them, they let frozen models repeatedly revise a persistent store of tools, skills, strategies, and memory, and score each version on unseen games.

Key Contributions

1. A controlled protocol for persistent self-improvement. Model weights stay frozen; each acting episode starts in a fresh context; the only thing carried forward is a self-constructed artifact inventory (executable Python tools, natural-language strategies/skills, and textual memory). Proposed revisions must pass structural and executable checks before becoming persistent, and rejected attempts still count toward cost.

2. The "agent plasticity" metric. Plasticity is defined as held-out performance gain per unit learning cost, reported in score points per $1,000. The headline variant, P_sat, fits a saturating curve to held-out in-distribution (ID) scores and measures gain per unit cost up to the cost of reaching 90% of the fitted rise.

3. A three-way failure taxonomy for the improvement loop. For each identified decision failure, the authors classify whether no covering artifact existed, whether one existed but was unused, or whether one was used and the failure remained — separated from decision-level reuse rates.

4. An empirical comparison across eight-plus frontier models and four games. The paper reports full learning curves on training, held-out ID, and held-out out-of-distribution (OOD) splits rather than endpoints alone, showing that capability and acquisition efficiency diverge.

Main Findings

• Improvement is model- and task-dependent. In Hard chess, Claude Fable 5 and Claude Opus 5 raise held-out ID scores from 37.5% and 25.0% at checkpoint 0 to 73.3% and 66.4% averaged over checkpoints 16–20. GPT-5.6 Sol starts near 0% and reaches 36.9%. Several other models stay close to their initial performance or decline, and most NetHack lineages show modest or negative early-to-late changes.

• Endpoint capability is not acquisition efficiency. On the combined three-game curve (chess Hard and Easy, Go, Hex), GPT-5.6 Sol has the highest estimated P_sat at 298 percentage points per $1,000, followed by Claude Opus 5 at 233, Claude Fable 5 at 57, and Claude Opus 4.8 at 14; GPT-5.6 Luna shows no reliable rise and is assigned P_sat = 0, while GPT-5.5 was still improving slowly so its saturation could not be established. Claude Fable 5 reaches the highest fitted performance level, so the best final performer is not the most plastic one. GPT-5.6 Sol starts below Claude Opus 4.8 yet exceeds it in both eventual held-out performance and plasticity.

• Transfer beyond the training regime is partial. Gains against stronger held-out OOD opponents are generally smaller or less consistent than held-out ID gains. In Go, Claude Fable 5's held-out ID score rises from 20% to 80% and its OOD score from 0% to 50% between checkpoints 0 and 20; in Hex, GPT-5.6 Sol goes from 40% to 77.5% on ID and from 28.1% to 78.1% on OOD.

• Task rankings do not transfer. Claude Opus 4.8 loses ground in Go while improving substantially in Hex, and Claude Opus 5 improves substantially in Go before a tool-making error at the final checkpoint causes a sharp drop. In NetHack, Claude Opus 5.5's mean score grows from about 2k over checkpoints 0–2 to about 6k over checkpoints 36–40, saturating at checkpoint 11 with P_sat = 61.75 normalized score points per $1,000; Claude Opus 5 rises from about 1.5k to 1.9k and Claude Sonnet 5 from about 680 to 800, Claude Opus 4.8 is roughly flat (about 700 to 730), and GPT-5.6 Luna (about 210 to 120) and GPT-5.6 Sol (about 570 to 420) decline.

• Low artifact reuse accompanies weak improvement. Pooled over Hard and Easy chess, GPT-5.6 Luna takes about 98% of relevant decisions without artifact use, Gemini 3.1 Pro about 87%, Claude Opus 4.6 about 51%, and GPT-5.5 about 28%, compared with 2–6% for Claude Fable 5, Claude Opus 5, Claude Opus 4.8, and GPT-5.6 Sol. Across 19 model–game cells in Chess (Hard), Go, and Hex, held-out ID reuse correlates with held-out ID score gain from checkpoint 0 to checkpoints 16–20. GPT-5.6 Luna's reuse rate in Go is about 1%.

• Reuse is not a proxy for capability. Claude Fable 5, Claude Opus 5, Claude Opus 4.8, and GPT-5.6 Sol all reuse artifacts at 94–98% of relevant chess decisions yet attain substantially different scores. The strongest reusers fail less often overall (about 0.16 versus 0.38 identified failures per relevant decision for the other models), but 83–99% of their remaining failures occur while a relevant artifact is in use.

• Reuse can rise without performance rising. In Hard chess, GPT-5.6 Sol and Claude Opus 4.8 go from below 20% reuse at checkpoint 1 to 93–99% at checkpoint 3, but their held-out ID scores peak only at checkpoints 12 and 5. In Go, Claude Opus 4.8 raises its share of relevant decisions with successful reuse from 59–65% to 93% while its held-out ID score falls from 40% at checkpoint 0 to 10–25% over checkpoints 15–20. Claude Opus 4.6's share of held-out ID failures occurring despite artifact use rises from 6.5% (checkpoints 1–5) to 52% (16–20) while failures per relevant decision stay flat (0.42 to 0.41), and GPT-5.5 uses artifacts at 96–100% of relevant decisions at checkpoints 2–8 but only 32–50% from checkpoint 9 on, with unchanged artifacts.

• Relevance is not coverage. Relevant artifacts are identified in at least 99% of decisions in 25 of 27 lineages, yet 67–100% of absent-artifact failures in Go and Hex occur at decisions that had a broadly relevant artifact available.

Methodology in Plain English

The model is frozen — nothing about its weights changes. Instead, each model keeps a "notebook" of things it has built for itself: small Python programs, written strategies, and notes. Every game starts with a blank conversation, so the notebook is the only memory that survives.

The loop works like this. The agent plays a batch of training games using the current notebook. Then it reads back the trajectories, outcomes, errors, and move-quality feedback, and proposes edits to the notebook. Edits must be structurally valid and actually run before they are saved; if a proposal is rejected, the old notebook stays. Then the agent plays again: 16 training games, 12 held-out ID games, and 12 held-out OOD games per checkpoint in Hard chess, over 20 update rounds. Held-out trajectories and scores are never shown to the reflection step, so any held-out gain must come from artifacts that generalize.

Chess uses Stockfish capped at 20,000 search nodes per move, and scores are 100(W + 0.5D)/N; unfinished games count as losses. NetHack runs to checkpoint 40 with ten fresh unseen games per checkpoint. Cost is tracked via token usage expressed in dollars, including rejected reflection attempts, and plasticity is that cost divided into the held-out score gain. For P_sat, the authors fit a saturating curve with four jointly fitted parameters, assign P_sat = 0 if the fitted 90% rise is under 10 percentage points, and extrapolate up to three times the final observed cost when no checkpoint reaches the threshold.

For the failure analysis, the authors combine semantic judgments by Gemini 3.1 Pro with execution evidence (file reads and persistent Python invocations) to decide whether an artifact was relevant and whether it was materially used. They stress that this is observational: use does not prove causal effectiveness.

Why This Matters

Impact on research. Benchmarks that report a single score conflate "was already good" with "got better." This paper separates the two and shows they can rank models in opposite orders — a direct challenge to leaderboard-style evaluation of agentic systems. It also reframes plasticity as a possible post-training target rather than a fixed property.

Real-world applications:

• Coding and software agents that accumulate reusable scripts and conventions across sessions — the Claude Code and Codex setups in NetHack are close to this deployment pattern.
• Long-running customer-support or operations agents where the value depends on whether past tickets turn into durable, generalizable procedures.
• Scientific or analytical assistants that build up domain tooling and heuristics over repeated campaigns.
• Game-playing and simulation agents where self-authored engines and search heuristics are natural artifacts, as in the Claude Fable 5 chess-engine example.

Industry relevance. Cost per unit of improvement is a budget question. The finding that the best final performer is not the most cost-efficient learner (Claude Fable 5 versus GPT-5.6 Sol) matters directly to teams deciding which model to run in a long-lived agent loop, and the failure taxonomy gives teams a diagnostic for whether to invest in retrieval or in artifact quality.

Future Directions

• Causal interventions on artifacts. The authors propose artifact removal, replacement, and forced invocation to separate "the artifact was bad" from "the agent applied it badly," since the current diagnostics are observational.

• Disentangling feedback from compute. Frozen weights and fresh contexts isolate artifacts, but they do not separate the value of environment feedback from the extra computation spent analyzing it. Budget-matched reflection without feedback and controlled actor–artifact comparisons are suggested.

• Making plasticity a training objective. Reinforcement learning could reward not just immediate task performance but the ability to construct, retain, and effectively use knowledge that improves future behavior — shifting post-training from "capable at deployment" toward "fast at becoming capable."

• Cross-task transfer and better measurement. Separately trained lineages do not establish that artifacts or improvement strategies transfer across tasks, and P_sat depends on score headroom, horizon, saturation-fit assumptions, and provider pricing — motivating full curves alongside scalar measures.

Target Audience

Researchers and engineers building long-lived or self-improving agents, evaluation and benchmark designers, and teams deploying agent harnesses such as Claude Code or Codex in production. It is also useful for continual-learning researchers who want a non-parametric, cost-normalized view of adaptation, and for anyone who reads agent leaderboards and wants to know what a single score hides. Readers need only moderate background: comfort with held-out evaluation and learning curves is enough, since the protocol itself is described in plain operational terms.

Authors’ abstract

AI agents increasingly operate in environments where they can diagnose failures and improve through experience, yet existing evaluations largely measure what an agent can do at a fixed point in time rather than how effectively it learns. Evaluating self-improvement requires answering three questions: does future performance improve and generalize beyond the interactions that enabled learning; how efficiently are new capabilities acquired; and where does the self-improvement process break down? To answer these questions, we study self-improvement in a controlled setting where agents amortize past experience into reusable artifacts that are inherited by future instances. At each checkpoint, we measure performance on training and held-out environment interactions while accounting for learning cost. We introduce agent plasticity, the efficiency with which an agent converts experience into gains in future held-out performance. Across multiple environments, frontier models exhibit sharply different improvement trajectories despite comparable opportunities to learn. Some achieve substantial and persistent gains, while others remain near or below their initial performance, and gains within the training regime often transfer only partially to out-of-distribution conditions. Endpoint capability and acquisition efficiency also diverge: the agent that ultimately performs best need not be the one that improves most efficiently. Tracing failures through the improvement loop further reveals different candidate bottlenecks. Agents with low plasticity often fail to reuse relevant artifacts, whereas more plastic agents may still fail despite reusing relevant artifacts, pointing to limitations in artifact quality, generalization, or application. Evaluating self-improving agents requires measuring not only what they can do, but how effectively they become better through experience.