Harness-Zero: Harness Distillation via Agent-as-Harness

AI summary

Overview

Research area: LLM agents, specifically agent harness design and model distillation (training-time transfer of harness-induced behavior into model weights).

Technical level: Intermediate to Advanced. The core ideas are intuitive, but the training setup, notation, and evaluation details assume familiarity with agent loops, supervised fine-tuning, and harness/tool abstractions.

Scope (one sentence): The paper proposes Harness-Zero, a three-stage framework that distills the behavior of an optimized agent harness into a model's parameters through an "agent-as-harness" review process, so the harness gains persist under a single minimal target harness at deployment.

What This Paper Is About

An LLM agent's performance depends on both the model and its harness: the external system that organizes tool use, manages context and state, and controls environment interaction. Harness optimization improves performance, but those gains stay tied to the harness at deployment, and because the best harness varies across domains, instances, and base models, a general-purpose agent must either settle for a suboptimal shared harness or route among an ever-growing set of specialized ones. The paper studies agent harness distillation: using a domain- or instance-optimized harness as training-time guidance and transferring the behaviors it induces into model weights, so its gains survive under a single fixed target harness.

The stated obstacle is that the source and target harnesses differ in action space and available information, so trajectories collected under the optimized harness cannot be used directly as supervision for the target harness.

Key Contributions

1. Formulates agent harness distillation and presents Harness-Zero, which transfers behaviors induced by an optimized harness into model parameters for deployment under a fixed target harness.
2. Introduces agent-as-harness, which translates guidance from an optimized harness into executable supervision at the student's response boundary, enabling imitation learning across harnesses with different action spaces.
3. Shows experimentally that agent-as-harness can outperform code-as-harness on frontier models (81.1% vs. 78.1% averaged across six benchmark–model settings), that Harness-Zero retains optimized-harness gains after the harness is removed, and that the distilled model recovers harness-induced behaviors.
4. Provides controlled ablations comparing harness-guided review against alternative supervision sources (teacher trajectories, trajectories under the evolved harness, review with an empty reference harness, and review with oracle answers).

Main Findings

• Agent-as-harness beats code-as-harness on average: Across six benchmark–model settings, agent-as-harness with the adapted reference harness K averages 81.1%, versus 78.1% for meta-harness and 68.6% for mini-SWE-agent. With an empty K it averages only 69.2%, so review alone explains little of the gain. Relative improvement over the target harness h was +22.5% for meta-harness and +27.6% for agent-as-harness with K.

• Distillation raises the base model's macro-average from 23.3% to 44.3%: That is a 21.0-point absolute gain and a 90.1% relative improvement over the base Qwen3.5-9B under h. It exceeds the 41.7% macro average the base model reaches with the evolved harness h* still attached.

• General-purpose harnesses do not help the base model: DeepAgents averaged 20.5% (a change of −2.8 points versus the 23.3% base) and Claude Code averaged 15.9% (−7.4 points). Per domain, DeepAgents scored 35.0 / 19.6 / 7.0 and Claude Code scored 31.0 / 10.7 / 6.0 on SpreadsheetBench / AppWorld / USPTO. The paper attributes this to a 9B model handling larger generic tool suites and extended context poorly, and to USPTO and AppWorld requiring domain-specific tooling and constraints.

• Procedural harness behavior is easier to internalize than deep domain knowledge: On SpreadsheetBench (44.0 vs. 39.0) and AppWorld (58.9 vs. 48.2), Harness-Zero under h alone surpasses h*. On USPTO it improves over the base model (30.0% vs. 12.0%) but trails h* (38.0%), where h* supplies reaction priors, candidate-generation logic, and executable SMILES validation.

• Stronger demonstrations alone are not enough: Fine-tuning on GPT-5.6 Sol trajectories leaves the student at the 12% base result on USPTO, even though that source succeeds on 52.0% of the 500 collection tasks. The base student's own trajectories under h* also return 12% after SFT.

• Answer access produces worse distillation than procedural guidance: Review with an empty K reaches only 11% pass@1. Providing oracle answers raises collection success to 98.6%, yet the distilled model reaches only 15%. K contains no task answers and reaches a lower collection success of 59.4%, but its trajectories yield 30% pass@1.

• Executing the evolved harness during collection does not make its behavior transferable: Mounting h* improves GPT-5.6 Sol's collection success from 52.0% to 62.0%, but the resulting distilled student falls from 12% to 3%. Models distilled from teacher trajectories under h* extensively attempt unavailable harness-tool calls, and many trials exhaust the turn limit, which the paper attributes to action-space mismatch.

• Harness-induced behaviors are recovered at 82.3% on average: Across 28 harness-exclusive patterns in three domains (18 on SpreadsheetBench, 6 on USPTO, 4 on AppWorld), Harness-Zero recovers 82.3% on average. Recovery spans memory, skill, tool, and middleware sources. Examples include 100% recovery for "Avoid fragile coordinates" (19/19), "Inspect before editing" (32/32), "Validate with RDKit" (60/60), "Canonicalize SMILES" (73/73), "Save requested workbook" (14/14), and "Reload saved workbook" (17/17); the lowest is 40% for "Retrieve complete pages" (21/53). The base model scores 0% on every pattern by construction, since patterns were selected only where the behavior appears under h* and never under h.

• Costs and limits: Reviewing every proposal requires an additional model call per step, raising mean USPTO latency to 2.4× during trajectory collection; the overhead is absent after distillation. The method also depends on a capable harnessing model, and with weaker models review can become harmful, with agent-as-harness losing its advantage over code-as-harness (reported in Appendix F).

Methodology in Plain English

The framework has three stages.

1. Evolve and adapt. Starting from rollouts on training tasks under a fixed minimal target harness h (a mini-SWE-agent-style harness with one Bash execute tool), the authors evolve a student-side harness h* over three rounds. Each round, a swarm of analysis agents (default 10 coder subagents per the evolution skill) examines tasks that failed the previous round and proposes reusable changes to tools, middleware, skills, or memory; a main evolution agent consolidates them. The resulting h* follows the DeepAgents abstraction. The evolved harness is then adapted into a private reference harness K for a separate harnessing agent: tools become specifications for student-native equivalents, student-side middleware becomes review middleware that privately alerts the harnessing agent, and skills and memory become diagnostic criteria and intervention guidance.

2. Collect trajectories (agent-as-harness). The student runs under the target harness h and proposes a response at each turn. A harnessing agent reviews each unexecuted proposal against K, its own private history, and the student-visible context, and either passes it or replaces it with the smallest coherent correction expressed as a complete response valid under h. Only the accepted response enters the student's visible trajectory and is executed through h; the rejected proposal and the review discussion stay private. The harnessing agent can read K but cannot inspect hidden solutions or verifier feedback, nor environment state outside the student-visible context, so any additional evidence must be obtained by proposing an action available under h (for example, replacing a premature completion with code that checks the student's work).

3. Train and deploy. Supervised fine-tuning is applied to the accepted responses in the reviewed rollouts, including unchanged student proposals and harness-guided replacements, with any reviewer-perspective reasoning masked from the loss. At deployment the system is the distilled policy together with h alone, without h*, K, or the harnessing agent.

Setup details. Three domains were used: SpreadsheetBench Verified (400 real-world spreadsheet tasks; 300 for harness evolution and training data collection, 100 held out), AppWorld (official train and development sets merged into a 147-task training split; 168 test_normal tasks held out, grouped into 56 three-task scenarios), and USPTO Retrosynthesis (500-task training split balanced across ten reaction classes; disjoint 100-task test split). All three run in the Harbor framework. Metrics are pass@1 for SpreadsheetBench and USPTO and scenario goal completion (SGC) for AppWorld. Training-free experiments used GPT-5.6 Sol and DeepSeek-V4-Pro; distillation used Qwen3.5-9B as base model with GPT-5.6 Sol as the harnessing agent; harness evolution used Kimi K3 under Kimi Code. Reasoning was enabled for all models with reasoning effort set to high when applicable. Training data comprised 487 rollouts for SpreadsheetBench, 282 for AppWorld, and 500 for USPTO; LoRA SFT ran for two epochs using the Tinker recipe.

Why This Matters

Impact on research. The paper reframes harness engineering as a source of training signal rather than only an inference-time scaffold. It argues that the advantage of agent-as-harness over fixed code harnesses should widen as foundation models improve, because the harnessing agent interprets guidance against the current trajectory instead of encoding stale assumptions about how a model should act. It also complements prior work on harness internalization (EvoHarness-RL on ALFWorld, which retains the external workspace at deployment) and on privileged workflow distillation (OPHSD, which does not involve an interactive agent loop), and it responds directly to a controlled coding-agent study finding that changing the evaluation harness affected performance more than the training method, and that training with feedback collected across harnesses did not improve transfer to a held-out minimal ReAct harness.

Real-world applications (as suggested by the paper's domains and claims):

• Spreadsheet automation, where recurring procedures such as inspecting before editing, avoiding fragile coordinates, protecting pre-filled cells, and verifying the saved output can be internalized rather than re-enacted by external scaffolding.
• Multi-application tool use, where a model must decide when to take an action versus answer, retrieve complete pages, and avoid repeated failed calls across simulated applications.
• Scientific reasoning workflows such as single-step retrosynthesis, where harness-provided validation (RDKit checks, SMILES canonicalization, candidate enumeration) is partially transferred into the model.
• Agent deployment in resource-constrained settings, where running a single minimal harness is cheaper than maintaining and routing among a growing collection of specialized harnesses with recurring context, model call, tool call, and orchestration costs.

Industry relevance. The work targets the practical tradeoff between a suboptimal shared harness and a fragmented collection of specialized ones, and it offers a path where harness development accumulates in model parameters instead of remaining split across external scaffolds. It also quantifies a training-time cost (a 2.4× mean USPTO latency increase during collection) that disappears at deployment, and it identifies a dependency on a sufficiently capable harnessing model.

Future Directions

• World models for harnessing decisions. A harnessing agent faces a counterfactual prediction problem: it must anticipate how executing the student's proposal would change the environment and whether intervening would produce a better trajectory. The authors propose training stronger agent world models and specializing their predictive capabilities for harnessing decisions.
• Selective and finer-grained intervention. The current design reviews every proposal and restricts intervention to passing or replacing the full response, which incurs unnecessary calls on sound proposals and offers only coarse-grained control. Candidate improvements include proxy signals to invoke review selectively, token insertion, and latent-space steering.
• Better training objectives. Each replacement pairs a rejected and a preferred response at the same state, so preference learning could exploit comparison signals that response-level SFT discards.
• Open questions on completeness. Some harness mechanisms, such as context management, are not fully expressible as student responses out of the box, and SFT may fail to fully internalize deep domain knowledge encoded by the evolved harness; whether broader pretraining or mid-training coverage, or more distillation trajectories, resolves the domain-knowledge gap is left open.

Target Audience

Researchers and engineers working on LLM agents, agent harnesses, and tool-using systems; practitioners interested in distillation, supervised fine-tuning, and transferring inference-time scaffolding into model weights; and readers tracking work on model–harness co-evolution, strong-to-weak transfer, and recursive self-improvement. Readers should be comfortable with agent loops, harness abstractions (tools, middleware, skills, memory), and evaluation protocols such as pass@1 and scenario goal completion.

Authors’ abstract

Agent harnesses, the external systems that mediate model-environment interaction, can substantially improve agent performance, but their gains remain tied to the harness at deployment. Because the best harness varies across domains, instances, and models, a general-purpose agent must either settle for a suboptimal shared harness or route among an ever-growing set of specialized ones. We therefore study agent harness distillation: using a domain- or instance-optimized harness as training-time guidance and transferring the behaviors it induces into model weights, so that its gains survive under a single fixed target harness. The challenge is that the two harnesses differ in action space and available information, so guidance from the optimized harness cannot serve directly as supervision for the target one. We introduce Harness-Zero, which enables harness distillation through agent-as-harness. Guided by the optimized harness, a harnessing agent corrects student responses before execution in the target harness's action space, turning harness guidance into training demonstrations. Fine-tuning on the resulting trajectories internalizes harness-induced behavior into the model, so the specialized harness can be removed at deployment. Our experiments spanning knowledge work, tool use, and science domains show that: (1) For frontier LLMs using the same evolved harness, agent-as-harness outperforms code-as-harness. (2) With the specialized harness removed at deployment, Harness-Zero improves the base model's macro-average task success from 23.3% to 44.3%, even exceeding the 41.7% it reaches with that harness still attached. (3) Harness-Zero recovers harness-induced behaviors absent from the base model, with 82.3% average recovery across 28 patterns in the three domains.