Aspire: Can Models Self-Evolve from Vague Goals?

AI summary

Overview

Research area: Large language model self-evolution, autonomous post-training, and agent benchmarking.

Technical level: Advanced. The paper assumes familiarity with post-training pipelines (SFT, GRPO, LoRA), checkpoint selection and rollback, agent harnesses, and benchmark construction with sealed evaluation sets.

Scope: The paper introduces Aspire, a benchmark and interactive environment that asks whether an LLM agent given only a vague natural-language capability goal, with all downstream evaluation tasks hidden, can operationalize that goal into data, training updates, and validation signals, and produce a real capability gain in either its model weights or its agent harness.

What This Paper Is About

Existing work on LLM self-evolution, such as PostTrainBench, LaMDAgent, Evo-Memory, and SEAL, starts after a human has already turned a broad capability request into an explicit task, metric, and success criterion, so the agent only searches over how to optimize a fixed objective. Aspire removes that scaffolding: the agent receives only a vague capability direction and must decide what to optimize, how to optimize it, and how to verify improvement, while the official evaluation items stay hidden. The goal is to test whether self-evolution still works when the agent, not the human, must build the learning signal.

Key Contributions

1. Formalization of target operationalization. The paper frames the conversion of a broad capability goal into trainable objectives, learning signals, and validation criteria as a missing axis of autonomous post-training, distinct from optimizing an already-specified objective.
2. A benchmark with sealed evaluation and a minimal interactive environment. Aspire provides only a natural-language goal, with a hidden expert-authored evaluation set of 520 items across six goals, and a unified agent-facing tool that supports both model-weight and agent-harness evolution while the controller handles data management, job scheduling, resource isolation, and checkpoint verification.
3. Outcome and trajectory evidence across three research questions. The paper reports RQ1 (vague goals versus explicit tasks with matched starting models, update tools, and compute budgets), RQ2 (self-directed weight updates from an instruction-tuned checkpoint), and RQ3 (harness evolution with fixed weights).
4. Identification of concrete failure modes. These include training on mismatched data, trust in narrow self-evaluation, regression erased by continued search, and answer-format collapse associated with numeric-label SFT.

Main Findings

• Vague goals change where search effort goes and lower aggregate scores. Claude Opus 4.8 under vague-goal prompting reaches 27.07 versus the 32.90 official Claude Opus 4.8 Max reference from PostTrainBench; GPT-5.6 reaches 29.58 versus the 36.23 official GPT-5.6 reference. Task-level changes are not uniform: Claude Opus 4.8 under vague-goal prompting is higher on HumanEval and BFCL, and GPT-5.6 under vague-goal prompting is higher on GSM8K.

• Vague goals shift resource use, not necessarily total action count. In matched Claude Opus 4.8 runs, vague-goal trajectories record 2,109 more seconds of decision-model thinking and 0.61 more GPU-idle hours per matched run pair, while active training and evaluation time falls by 1.27 hours. Accesses to agent-visible task materials and proxy evaluation scripts become 2.98 times and 2.39 times as dense. In a separate trace-level sample spanning the six shared settings, mean tool actions do not increase (188.5 to 179.2), while LoRA is used more often under vague goals.

• Strong explicit-task systems show different feedback profiles. Claude Opus 4.8 under vague-goal prompting performs 3.54 evaluations for 6.17 training starts, or 0.57 evaluations per start. Official GPT-5.6 performs 39.21 evaluations for 14.91 starts, or 2.63 per start, 4.6 times the feedback density. Official Claude Opus 4.8 Max has 2.89 evaluations and 5.14 training starts, or 0.56 per start, nearly identical to the vague-goal Claude Opus 4.8, but records 165,851 thinking characters versus 30,895 for official GPT-5.6.

• Final-only weight evolution rarely beats the base model. Across 24 runs (Qwen3.5-4B Self and Qwen3.5-9B Self, each optimizing all six goals in two separately executed terminal runs, reported as Avg@2 within a 40 GPU-hour budget), Qwen3.5-4B has 0/6 mean outcomes above its base score and Qwen3.5-9B has 1/6. The sole positive pair is scientific and academic reasoning, where both runs rise from 45.33 to 48.00, a two-run mean 2.67 points above the base score, and the only positive mean among all 12 model–goal pairs. At the individual-run level, 3/24 final checkpoints exceed their base scores, while rollback retains the base model in the other 21 runs.

• Adaptive feedback produces trained checkpoints readily but above-base scores rarely. Across 30 configuration–goal cells, 28 produce an evaluated checkpoint and 21 produce an eligible checkpoint. Only two cells have a best evaluated checkpoint above the base score: Qwen3.5-4B Self raises science from 44.00 to 45.33, but no Self cell retains a model above its base score, and Terra raises Qwen3.5-4B mathematics from 17.86 to 20.10, the only cell whose checkpoint scores above base, passes eligibility, and survives rollback. The authors note that without a separate confirmation slice this retained score is evidence of feedback-guided selection rather than an independently replicated capability gain.

• Apparent progress can be recovery from training-induced regression. The 22 cells with multiple evaluated checkpoints contain 62 consecutive transitions: 28 rise, 13 tie, and 21 fall. Later search beats the first evaluated checkpoint in 14/22 cells. For example, Qwen3.5-4B Self improves mathematics from 0.79 to 1.75 to 2.38 while remaining far below its 17.86 base score.

• More search does not mean better search. Using the same Qwen3.5-4B base model, Luna is highest on humanities (7.27) and logic (20.39), Terra on mathematics (20.10) and writing (16.01), and Sol on science (30.67) and medicine (11.00). In five of six goals, even the highest of these remains below the base score. Sol searches most broadly, producing 33 evaluated checkpoints with 76.56 settled training GPU-hours, yet none exceeds its corresponding base score. Terra's later mathematics descendants fall from 20.10 to 2.78 before recovering to 18.17.

• Blind continuation can end in poor final checkpoints. Three final-only runs continue for 16–20 training jobs and consume 26.570–32.286 GPU-hours, yet their submitted final checkpoints score only 0.159 on 4B mathematics, 2.540 on 9B mathematics, and 0 on 9B writing.

• A documented answer-format collapse. In the adaptive-feedback protocol, Qwen3.5-4B Self uses GSM8K or Hendrycks mathematics data in 30/32 dataset-import events, including runs targeting science, logic, and writing. In the final-only protocol, five submitted checkpoints are trained on numeric-label MMLU SFT: all 21,000 training targets are single-digit labels, all 279 corresponding evaluation outputs are single digits, and their scores are 0, 0, 0, 6.141, and 0. All 24 final evaluations complete without item errors.

• Harness evolution does not reach the engineered reference. With Qwen3.5-4B fixed as the runtime model and the original Qwen-Agent as reference, the Qwen3.5-4B Creator produces no valid successor harness and is assigned zero by the predeclared failure rule. All three valid successors are numerically below the reference Qwen-Agent, which obtains 28.64 task macro and 27.65 example micro. The Sol-created successor is closest at 27.22/25.97, 1.42 task-macro points below, followed by the Terra-created successor at 20.76/20.14 and the Luna-created successor at 19.32/18.33.

Methodology in Plain English

The researchers build a setting in which an agent is told only a broad capability direction, such as "become better at scientific reasoning," and is never shown the test items used to judge it.

Evaluation is kept on the controller side. The hidden set contains 520 expert-authored items across six goals: scientific and academic reasoning (75 items), humanities and social-science knowledge (110), health and medical reasoning (100), mathematical reasoning (126), logic, reliability, and instruction following (89), and academic and scientific writing (20). GPQA, MMLU-Pro, and MedQA are used only as references for task format, domain coverage, and approximate difficulty; no item from them is copied, rewritten, or included. Candidates pass independent review, blind multi-model difficulty calibration, exact and semantic deduplication, overlap auditing, scorer binding, and immutable versioning. Each goal is scored on its own slice rather than pooled, and the fifth goal is treated as one integrated reliability objective rather than three independently identifiable targets.

The agent interacts through a single tool with typed actions: search for, download, or import datasets; synthesize and register data; launch SFT or GRPO with LoRA or other permitted configurations; query job state; run checks on its own validation data; and branch or stop a lineage. The controller absorbs credentials, storage conventions, distributed-job mechanics, and recovery. Datasets registered for training are checked for overlap with the hidden evaluation set before admission.

Two evolution surfaces are studied, with the component under study changing while the decision model, controller, and evaluator remain fixed within a round. Weight evolution (RQ1–RQ2) varies model weights; harness evolution (RQ3) varies the runtime instructions, tool policy, workflow, memory, and validation logic around a fixed model. Safe retention works through score-gated rollback: the controller keeps the selected checkpoint only when its raw score change is positive, otherwise it reverts to the incoming state, which is why retained improvement cannot be negative by construction. The authors explicitly state this is a selection outcome, not evidence that every attempted update improves capability.

Two protocols separate open-loop from feedback-guided search. Under final-only, the agent may train multiple checkpoints but receives no score before submitting one terminal checkpoint. Under adaptive feedback, a bounded number of queries may evaluate the same hidden items for a goal, returning only aggregate scores and never items, answers, routing labels, rubrics, candidate outputs, or judge traces. The paper's content as provided is truncated during the RQ3 results table, so the authors' subsequent discussion, limitations, and conclusion are not available here.

Why This Matters

Research impact. The paper argues that existing self-evolution benchmarks presuppose the very work that dominates real deployment: turning a vague need into objectives, validation signals, and a working system. By isolating target operationalization and keeping evaluation hidden, Aspire shifts measurement from "can the agent optimize a given objective" to "can the agent decide what to optimize and verify it." It also supplies trajectory evidence that rising within-lineage scores can be recovery from regression rather than genuine capability gain, which matters for how self-improvement results are reported.

Real-world applications.

• Deploying a general-purpose model into a customer or domain environment where the need is expressed informally and no labeled task exists yet.
• Automating post-training for capability gaps that current benchmarks no longer reflect, instead of waiting for humans to author new benchmarks and rewards.
• Building or maintaining the agent harness around a fixed model, covering tool policy, workflow, memory, and validation logic.
• Constructing internal validation and checkpoint-selection pipelines when outcome feedback is sparse, delayed, or aggregate-only.

Industry relevance. The paper connects the setting to the forward-deployed engineer (FDE) role popularized by Palantir and adopted by frontier-model companies, whose success criterion is whether a deployed system is genuinely used, continues to work, and improves in response to failures. It cites the observation that a capable model is not yet a working system and that today the gap is largely closed by human engineers. The results show the current limits of that substitution: agents complete training and harness-editing loops routinely, but weight-level gains remain sparse and unstable, and even the strongest evolved harness remains below the engineered Qwen-Agent reference.

Future Directions

• Confirming retained gains on a separate slice. The one retained adaptive-feedback improvement was selected using repeated aggregate feedback on the same fixed evaluation slice, so the authors call for confirmation rather than treating it as an independently replicated capability gain.
• Recursive decision-model replacement. Although the system can promote a verified trained descendant to the decision model in a subsequent round, all reported runs keep the initial decision model fixed, so handoff and multi-round recursion remain untested.
• Multi-step harness self-improvement. The experiment evaluates one-step harness generation; no run evolves H1 into H2, leaving recursive harness improvement open.
• Understanding and preventing training-induced regression. Open questions include why agents select mismatched data such as GSM8K or Hendrycks mathematics data for science, logic, and writing goals, why numeric-label SFT produces single-digit answers across all 279 evaluation outputs, and how to build validation signals that detect format collapse before submission.

Target Audience

Researchers and engineers working on LLM self-evolution, autonomous post-training, and agent evaluation methodology, along with practitioners who deploy models into environments where goals arrive informally. It is most useful to readers who already understand post-training mechanics and benchmark construction, since the paper's contribution is a measurement and environment design rather than a new training algorithm.

Authors’ abstract

Many important forms of human learning begin with a vague goal, such as "become a better physicist" or "improve at research." Learners must interpret the goal, identify capability gaps, decide how to learn, and determine whether they have actually improved. In contrast, existing work on LLM self-evolution typically begins with tasks and evaluation metrics specified by humans, reducing self-evolution to optimizing an explicit objective rather than deciding what and how to learn. We introduce ASPIRE, a benchmark for vague-goal-driven self-evolution. ASPIRE provides only a natural-language capability goal while downstream evaluation tasks remain hidden. The agent must operationalize the goal by choosing data and update methods, constructing training and validation signals, and deciding when to evaluate. ASPIRE supports both model-weight and agent-harness evolution in a unified interactive environment and evaluates the resulting systems on a hidden, expert-authored set of 520 items spanning six goals. Our experiments show that vague goals redirect search effort toward goal interpretation. Current agents routinely complete training and harness-editing loops, but weight-level gains remain sparse and unstable, and the strongest evolved harness remains below the engineered Qwen-Agent reference. Agents often train on mismatched data and trust narrow self-evaluations, so local gains fail to transfer to hidden evaluation and continued search and training can erase earlier improvements.