PILOT in the Loop: Live Self-Improvement for Long-Horizon Agents

AI summary

Overview

Research area: LLM agent architectures — specifically self-improving agent harnesses (supervisor–worker orchestration, live steering, and skill/memory evolution) evaluated on long-horizon agentic benchmarks.

Technical level: Advanced. The paper assumes familiarity with agent loops (ReAct-style), subagent delegation, harness/skill libraries, and benchmark evaluation protocols such as pass rate, verifier outcomes, and token-efficiency metrics.

Scope: The paper introduces PILOT, a supervisor–worker harness that makes self-improvement "live" by letting a separate supervisor redirect an in-flight worker and simultaneously distil reusable skills and memory from the running trajectory, evaluated across two frozen backbones and three benchmarks.

AllSpark Team. Published 2026-08-27 (arXiv:2608.26530v1 [cs.AI]); paper dated August 25, 2026; code at github.com/XiaoYang66/Pilot.

What This Paper Is About

Long-horizon agent runs produce experience that could improve both the run currently underway and future runs: successful attempts reveal reusable procedures, and failed attempts expose recurring failure modes. Most existing self-improvement methods only process that experience after a run ends, so they cannot recover the active run, and any lesson learned can only be validated in a later rollout. PILOT's goal is to make self-improvement live — using emerging experience to redirect the active run and update the persistent harness — by resolving an architectural gap: existing architectures either make one agent both execute and self-critique (splitting its attention and context), or delegate to subagents whose output the main agent only receives after they finish.

Key Contributions

1. Formulating live self-improvement. The paper defines live self-improvement as a closed loop that uses emerging experience both to redirect the active run and to update the persistent harness, rather than only updating after execution completes.
2. Introducing PILOT, a supervisor–worker harness. PILOT couples two mechanisms over a shared stream of execution experience: live steering (a separate supervisor redirects or aborts an active worker mid-run) and live self-evolution (procedures, project conventions, and failure modes revealed during execution are distilled into reusable skills and memory).
3. Cross-backbone, cross-domain evaluation. PILOT is evaluated with two frozen open-weight backbones (GLM-5.1 and Kimi-K2.6) and three long-horizon benchmarks (Terminal-Bench 2.0, SWE-bench Multilingual, SWE-bench Pro), in both a one-shot setting and a self-improvement setting, with the same frozen model serving as both supervisor and worker.
4. Trajectory and efficiency analyses. The paper reports analyses of task recovery, skill growth, and token efficiency, including a manual trace-level classification of successful runs aided by live steering and two representative case studies.

Main Findings

• Leaderboard placement: Across two frozen backbones and three benchmarks, PILOT ranks first in five of six backbone–benchmark configurations.
• Terminal-Bench 2.0 margins: PILOT outperforms counterpart harnesses by as much as 9.8 percentage points. It reaches 71.9% on GLM-5.1 (5.0 points above OpenCode at 66.9%) and 71.3% on Kimi-K2.6 (4.4 points above Pi at 66.9%); averaged across the two backbones it reaches 71.6%, 5.3 points above Pi at 66.3%, the strongest single-agent baseline by average. Terminal-Bench 2.0 contains 89 runnable tasks.
• Hard-task gains in one-shot evaluation: PILOT reaches 55.0% on hard tasks with each backbone, exceeding Pi by 5.0 points on GLM-5.1 (50.0%) and 6.7 points on Kimi-K2.6 (48.3%).
• Software-engineering benchmarks: On SWE-bench Pro, PILOT averages 59.9%, 4.4 points above Pi at 55.5%. On SWE-bench Multilingual, PILOT averages 72.7% versus Pi at 72.9%. With GLM-5.1, PILOT ranks first on all three benchmarks; with Kimi-K2.6, it ranks first on Terminal-Bench 2.0 and SWE-bench Pro and second on SWE-bench Multilingual.
• Self-improvement over iterations: PILOT's best observed pass rate on GLM-5.1 rises from 66.3% at iteration 0 to 80.9% (a 14.6-point gain); on Kimi-K2.6 it rises from 68.5% to 80.9% (a 12.4-point gain).
• Comparison with other harnesses in the self-improvement setting: From iteration 0 to each harness's best observed result, PILOT improves by 14.6 points, compared with 7.9 points for OpenCode and 2.3 points for Pi, all on the same frozen GLM-5.1 backbone and the same initial skill library.
• Skill-library growth: The number of GLM-5.1 skills grows from 62 to 83, and Kimi-K2.6 skills from 50 to 81 (a growth of 21 and 31 skills, respectively).
• Gains concentrate on hard tasks: GLM-5.1 first reaches its maximum at iteration 14 and Kimi-K2.6 at iteration 13. GLM-5.1 gains 2 additional passes on Easy, 6 on Medium, and 8 on Hard; Kimi-K2.6 gains 1, 7, and 12 additional passes.
• Token efficiency: Mean output tokens per evaluated task fall from 28.5K to 16.3K on GLM-5.1 (a 42.9% reduction) and from 41.9K to 22.1K on Kimi-K2.6 (a 47.4% reduction). Successful evaluations per million output tokens rise by 110.3% on GLM-5.1 and 134.0% on Kimi-K2.6 relative to iteration 0.
• Live steering is used more on harder tasks: Of successful Terminal-Bench 2.0 one-shot runs, the share classified as aided by live steering is 0.0% on Easy for both backbones, 1.1% (GLM-5.1) and 8.1% (Kimi-K2.6) on Medium, 6.1% and 19.7% on Hard, and 2.3% and 10.6% overall.
• Case studies show mid-run recovery: In winning-avg-corewars (Medium, Kimi-K2.6), a worker spent over twenty minutes tuning DAT-clear variants before the supervisor told it to stop testing synthetic opponents and use a published warrior source; the worker adapted Silk Warrior 1.3 from the public mbarbon/corewar-koth archive and passed thresholds of Stone 93%, Paper 94%, Vampire 84%, Snake 55%, and G2-Clear 53%. In torch-tensor-parallelism (Hard, GLM-5.1), the supervisor corrected bias being added once per rank in RowParallelLinear; the worker changed to F.linear(x, self.weight, None) followed by all_reduce and a single bias addition, passing all 13 verifier tests across world sizes 1, 2, and 4.

Methodology in Plain English

The researchers split agent self-improvement into two roles running on the same frozen model: a worker that executes the task, and a supervisor that watches over it. The supervisor is connected to the worker throughout the run through a two-way channel. The worker can send three kinds of events — a Notification (progress, an intermediate result, or a potential risk, with execution continuing), a Question (pausing until the supervisor replies), and a Result (delivered automatically by the runtime when the worker finishes). The supervisor can take two actions — Steer (queue guidance for the worker's next turn after inspecting the relevant part of the trajectory; the current turn finishes first) and Abort (interrupt a worker session that is no longer useful).

Because the supervisor's context stays separate from the worker's, it retains a view of the goal, recent events, and signs that a run is going off track, while execution detail and dead ends stay in the worker's context. When the supervisor spots a reusable procedure, project convention, or recurring failure mode in the live trajectory, it writes it into the harness's skill library or memory, producing an updated harness that every subsequently spawned worker loads — within the same episode or in a later one.

The implementation extends the Pi coding-agent runtime: the supervisor is an agent session and workers are spawned in-process as separate sessions, with the same frozen model serving both roles. Evaluation uses two settings. In the one-shot setting, every task starts from a fresh harness state, isolating whether live steering keeps the worker on track. In the self-improvement setting, Terminal-Bench 2.0 runs are organized into iterations (one iteration is a complete sweep over the benchmark tasks); at the start of iteration i, every run gets an isolated copy of the same shared harness state, and updates can only be created from the live trajectory and environment feedback — no benchmark evaluation signal or reward reaches the supervisor or worker. Only after an iteration finishes do verifier outcomes decide which updates carry forward: updates from successful runs are retained, updates from failed runs are not. All evaluated harnesses receive the same self-improvement instruction, differing only in harness-native skill and memory paths.

Protocol: thinking enabled and preserved, maximum generation length 32k tokens, other parameters at official defaults, each configuration run twice with the mean pass rate reported, every task in an isolated container sandbox, and Terminal-Bench 2.0 tasks capped at three hours wall-clock. Some tasks with environment conflicts are excluded identically across harnesses (43 SWE-bench Multilingual tasks and 198 SWE-bench Pro tasks are excluded because their pinned environments cannot bootstrap the JavaScript agent runtimes or their Node.js versions are below the required minimum).

Why This Matters

Impact on research. The paper reframes self-improvement as a timing problem rather than only a storage problem: existing post-hoc methods (reflection, judge-based evaluation, self-evolving harnesses) can preserve experience but cannot act on the run that produced it, and cannot immediately validate a lesson against the execution that revealed it. PILOT also names a concrete architectural gap — single-agent self-correction splits attention between execution and oversight, while subagent delegation generally cannot redirect a subagent mid-run — and shows a harness-level design that closes it. The reported token savings alongside accuracy gains suggest that reusable harness knowledge reduces repeated exploration rather than merely adding more inference.

Real-world applications.

• Long-horizon software engineering: repository-level repair and multi-file implementation work where an error early in a chain compounds before the final test run.
• Terminal and system administration: multi-step shell workflows where an agent must reach a system or engineering goal over many commands.
• Recurring team workflows: settings where a developer or team returns to the same harness for related tasks and wants skills from earlier work reused rather than rediscovered.
• Compute-constrained agent deployment: use cases where successful evaluations per million output tokens matters as much as raw pass rate.

Industry relevance. The evaluated scenario — explicitly instructing an agent harness to retain reusable skills across related tasks — mirrors how teams already try to accumulate internal playbooks. Because PILOT keeps the model frozen and improves only the harness, the approach is compatible with closed or fixed-weight deployments, and because the same model fills both roles, it does not require a second, stronger model to provide oversight.

Future Directions

• Cheaper iteration. The limitations section notes that iterative self-improvement repeats every task across many iterations, making additional benchmarks and backbones substantially more expensive than a single inference run; this cost currently limits evaluation to three benchmarks and two open-weight backbones.
• Broader model coverage. Extending to broader benchmark coverage and to proprietary models remains future work.
• Heterogeneous supervisor–worker pairings. The paper uses the same frozen backbone in both roles, reflecting common use but leaving heterogeneous pairings and their trade-offs among oversight quality, task performance, and cost unexplored.
• Scaling oversight to concurrent workers. The pseudocode allows workers to be spawned concurrently or later in the same episode, with every event and action tagged by worker index; how supervision quality holds up under multiple simultaneous workers is not characterized in the reported results.

Target Audience

Researchers and engineers working on LLM agent architectures, agent harnesses, and self-improving agent systems; practitioners building long-horizon coding or terminal agents who care about recovering failing runs and accumulating reusable skills; and benchmark-focused readers interested in how a harness — rather than a model — moves pass rates, token cost, and per-token success rates on Terminal-Bench 2.0, SWE-bench Multilingual, and SWE-bench Pro. Readers unfamiliar with agent loops, tool use, and verifier-based benchmark evaluation will need background reading first.

Authors’ abstract

Long-horizon agent runs generate experience that can improve both the current run and future work. Most self-improvement methods process this experience only after execution ends, so they cannot redirect the active run or immediately apply and validate lessons learned from it. We argue that self-improvement should instead be live, using emerging experience both to redirect the active run and to update the persistent harness. Existing agent architectures do not fully support this goal. Single-agent self-correction combines task execution and trajectory assessment within one context, while subagent delegation separates execution but typically cannot redirect an active subagent. We present PILOT, a supervisor-worker harness for live self-improvement through two coupled mechanisms: (1) live steering lets a separate supervisor redirect or abort the active worker during execution; and (2) live self-evolution distils procedures and failure modes revealed during execution into reusable skills and memory. Across two frozen backbones and three benchmarks, PILOT ranks first in five of six configurations. On Terminal-Bench 2.0, PILOT outperforms counterpart harnesses by up to 9.8 percentage points. In the self-improvement setting, PILOT gains 14.6 points with GLM-5.1 and 12.4 points with Kimi-K2.6. Mean output tokens fall by 42.9% and 47.4%, while successful evaluations per million output tokens rise by 110.3% and 134.0%, respectively.