EVISKILL: Grounding Skill Evolution in Replayable Evidence

AI summary

Overview

Research area: Artificial intelligence — LLM agents, continual skill evolution, and evidence-based verification of agent behavior.

Technical level: Intermediate. The paper assumes familiarity with LLM agent benchmarks, tool use, and iterative skill/prompt refinement, but its core argument (verify changes against the behavior that motivated them) is stated in accessible terms.

Scope: The paper proposes EviSkill, an evidence-driven framework that links proposed skill edits to replayable execution evidence, verifies them by re-execution, and carries supported edits and evidence across evolution epochs.

What This Paper Is About

LLM agents can improve without changing their model weights by accumulating reusable "skills" — external instructions for choosing tools, ordering actions, and respecting domain constraints. Existing methods derive these skill edits from execution trajectories, but a trajectory is local and context-dependent while a committed edit is meant to persist and generalize, so edits can overfit the observed case. EviSkill's goal is to make skill evolution evidence-driven: each proposed edit is explicitly tied to the execution context that justifies it, tested by re-executing that context, and retained or revised across epochs rather than being accepted or discarded wholesale.

Key Contributions

1. Diagnosis of experience-driven skill evolution. The authors identify two limitations: execution evidence motivating a revision is not explicitly maintained alongside the resulting revision, and evolutionary information generated in one round cannot be continuously reconsidered across epochs.
2. The EviSkill framework. An evidence-driven framework that organizes continual skill evolution into three stages — evidence construction (Evidence-Grounded Edit Synthesis), behavioral verification (Replay-Guided Edit Verification), and cross-epoch refinement (Cross-Epoch Evidence Propagation).
3. The Replayable Evidence Card. A representation C_i = <i, G_i, d_i> recording a proposed correction d_i, a persistent identifier i for tracking across epochs, and supporting trigger ranges G_i — intervals [s, t] in a named source trajectory from a source epoch, which can be replayed.
4. Empirical validation. Evaluation on three interactive benchmarks across multiple LLM backbones, showing consistent improvements over skill-evolution baselines, plus ablation and mechanism analyses showing that replay filters unsupported revisions and that cross-epoch propagation lets prior revisions and evidence support continued refinement.

Main Findings

• Broad performance gains from evidence grounding. EviSkill achieves the best accuracy in 14 of the 18 model–dataset settings shown and ranks among the top two methods in 16. It improves over NoSkill in all settings with an average gain of 17.93 percentage points.
• Large ScienceWorld margins. On ScienceWorld, EviSkill exceeds the strongest baseline by 15.64 points with Qwen3.5-9B and by 10.43 points with GPT-5.4.
• Evolving a skill does not guarantee improvement over the initial skill. On ScienceWorld with GPT-5.5, the initial LLM skill reaches 76.78%, while Trace2Skill, EvoSkill, and SkillGrad obtain 65.40%, 54.50%, and 67.30% respectively; EviSkill reaches 83.41% and improves over LLM Skill in 17 settings while matching it in the remaining one.
• Preliminary study 1 — experience-derived edits have mixed behavioral outcomes. Applying Trace2Skill edits to the original skill and re-executing their associated trajectory tasks produced diverse outcomes across all benchmarks: some improved task performance, others produced no improvement or degraded original behaviors.
• Preliminary study 2 — rejected revisions can contain useful edits. Selectively retaining edits from globally rejected revisions improved performance on the associated trajectory tasks compared with the previous validated skill.
• Replay rejects many candidate edits. Of candidate edits grounded in training trajectories, 20.7% on ALFWorld, 41.8% on AppWorld, and 42.5% on ScienceWorld were assigned to reflection or rejection at the initial replay.
• Reflection often recovers edits. Feedback-guided refinement recovered 13 of 16 reflected edits on AppWorld, 49 of 54 on ScienceWorld, and the single reflected edit on ALFWorld.
• Provisionally retained edits are often promoted later. Of the 96 edits provisionally retained after post-rejection replay, 46.9% entered the Validated Skill one epoch later, 14.6% two epochs later, and 11.5% three epochs later — 72.9% in total. The remaining 27.1% were still provisional when the four-epoch budget ended, so their eventual promotion remains unobserved.
• Evidence is reused across epochs. Across all benchmarks, 23.0% of unique Evidence Cards were selected for Evidence Windows in at least two distinct epochs, while 54.3% were selected in exactly one. Reuse varied by environment: 66.6% of ALFWorld Cards were pre-resolved, whereas AppWorld and ScienceWorld contained larger shares selected across multiple epochs.
• Ablation — replay matters. Removing Replay-Guided Edit Verification reduced average accuracy by 2.49, 4.56, and 2.85 percentage points on ALFWorld, AppWorld, and ScienceWorld respectively (averaged over the three GPT backbones).
• Ablation — cross-epoch propagation matters. Disabling Cross-Epoch Evidence Propagation reduced accuracy on all three benchmarks, with the largest decrease on AppWorld, from 89.68% to 85.52%.
• Where the advantage is weaker. The paper reports that the advantage over competing methods is less consistent on AppWorld.

Methodology in Plain English

The framework runs over a series of epochs. In each epoch, an action agent produces trajectories for the training tasks using a Working Skill, which is the current Validated Skill combined with a Provisional Edit Ledger of replay-supported edits that have not yet been permanently incorporated. Two sources of evidence are extracted: Trajectory Evidence Cards from the current trajectories, and Contrastive Evidence Cards from comparing trajectories of the same task across adjacent epochs for edits being tracked. These join carried-over cards to form an Evidence Pool.

Cards with compatible source contexts and correction intents are grouped into Evidence Windows, and an LLM editor turns each window into candidate edits, each retaining links to its supporting cards. Those links determine which trajectory intervals must be replayed.

For verification, the system reconstructs the execution state preceding each trigger range, re-runs the segment under the working skill plus the candidate edit, and has an LLM evaluator compare the replayed segment against the source segment. The evaluator returns accept, reflect, or reject. Reflected edits are revised with feedback and replayed again; only accepted edits form the consolidated candidate revision.

For cross-epoch handling, the candidate revision is compared against the current Validated Skill on a held-out validation set. If it wins, it becomes the new Validated Skill, the provisional ledger is cleared, and the incorporated edits become the next round's tracked edits. If it loses, the Validated Skill is unchanged, but each consolidated edit is replayed again — this time on top of the Validated Skill alone, without the provisional edits — and any that still pass are retained in the provisional ledger for future epochs. Cards move through lifecycle states: active (eligible for new edit synthesis), protected (supporting provisional edits, excluded from new windows but still usable for cross-epoch comparison), archived (supporting incorporated edits, likewise excluded), and stale (excluded after repeated replay rejection). After a fixed number of epochs, the latest Validated Skill is frozen for test-time inference, and unpromoted provisional edits are excluded.

Why This Matters

Impact on research. The paper reframes skill evolution as a process of establishing, testing, and revising the support for each change before committing it as reusable knowledge, rather than simply accumulating experience-derived edits. Its preliminary studies give direct evidence for two claims that challenge standard practice: an edit is not necessarily effective because it came from execution experience, and not necessarily ineffective because the revision containing it was rejected.

Real-world applications (drawn from the paper's benchmarks and cited domains):

• Application-API task execution, as tested on AppWorld.
• Scientific experimentation procedures, as tested on ScienceWorld.
• Household/embodied task execution in text-based environments, as tested on ALFWorld.
• Tool-use procedures, web navigation, and coding workflows, which the related work identifies as settings where agent skills are used.

Industry relevance. Because the approach modifies external skill artifacts rather than model parameters, it fits deployments where weights cannot be retrained — for example agent products that ship inspectable procedural instructions. The Replayable Evidence Card also provides a traceable link from each instruction to the execution context that justifies it, which matters for auditing and debugging agent behavior.

Future Directions

• Longer evolution horizons. 27.1% of the 96 provisionally retained edits were still provisional when the four-epoch budget ended, so whether they would eventually be promoted remains unobserved.
• Closing the AppWorld gap. The paper reports that the advantage over competing methods is less consistent on AppWorld, leaving room to understand which environment properties reduce the benefit of replay verification.
• Broader evaluation. Results here cover three benchmarks and six backbones, with full tables in the appendix; broader domains and backbones are untested in the provided content.
• Cost of replay. Verification requires re-executing trajectory segments, and the provided content does not report the computational or token cost of that re-execution, nor a comparison of its overhead against the measured accuracy gains.

Target Audience

Researchers and engineers working on LLM agents, agent memory, and external skill or prompt refinement; practitioners who need to improve agent behavior without retraining model weights; and anyone evaluating how to validate automated edits to agent instructions, who will find the Replayable Evidence Card and the three-way accept/reflect/reject replay decision useful as a concrete design.

Authors’ abstract

Continual skill evolution enables LLM agents to accumulate and refine reusable procedural knowledge from interaction experience without updating model parameters. Its effectiveness depends on determining not only what to change, but also why a change is justified and when it should become persistent guidance. However, existing experience-driven methods can lose the behavioral evidence and task contexts supporting edits. Moreover, a global validation outcome provides an incomplete judgment of its constituent changes: locally supported corrections may be discarded with a rejected revision, while evidence may require further experience to inform useful updates. To this end, we introduce EVISKILL, an evidence-driven framework that organizes execution observations into Replayable Evidence Cards and synthesizes edits with explicit links to their supporting contexts. Targeted replay verifies these edits through re-execution and provides feedback for correction. Across epochs, EVISKILL preserves evidence and provisionally retains supported edits for further refinement, while global validation governs their incorporation into the final skill. Experiments on three interactive benchmarks across six LLM backbones demonstrate the effectiveness of this approach.