SHE: Trajectory-driven Safety Harness Evolution for LLM Agents

AI summary

Overview

Research area: LLM agent safety, specifically the safety of the agent harness — the non-model layer that manages context, memory, tool access, permissions, and runtime control for tool-using agents.

Technical level: Intermediate to Advanced. The paper assumes familiarity with LLM agents, tool use, guardrails, attack-success-rate metrics, and prompt/artifact-based safety controls, though its central idea (treat the harness as something that can be edited and evolved) is explained accessibly.

Scope in one sentence: The paper proposes Safety Harness Evolution (SHE), a framework that decomposes an agent's safety harness into four editable artifacts and iteratively refines them from rollout trajectories, reporting improved safety and utility on Agent-SafetyBench and transfer to held-out AgentHarm.

What This Paper Is About

Most existing agent safety mechanisms are built once and then frozen at deployment, so they cannot automatically learn from the execution trajectories where they fail. A second problem is that safety responsibility is spread across coupled harness components, so when something goes wrong it is hard to tell which part should be fixed. SHE addresses both by splitting the harness into four functionally bounded artifacts and running an attribution-guided loop that turns trajectory failures into targeted, validated edits to those artifacts.

Key Contributions

1. An evolvable safety harness. SHE treats the harness itself as an object that learns from rollout trajectories, rather than as a fixed deployment artifact.

2. A four-artifact decomposition with explicit safety responsibilities. The harness is represented as System Prompt (global behavioral contract), Rule Bank (structured safety rules and interventions), Safety Memory (experience from failure cases that remain unresolved after repeated evolution attempts), and Tool Policy (tool authority and runtime enforcement) — giving each artifact a clear validation target.

3. An attribution-guided evolution loop. For each evaluated trajectory, SHE produces a structured risk diagnosis, routes it to the responsible artifact(s), generates bounded local edits, validity-checks them, and keeps a candidate harness only if it improves safety without degrading utility.

4. Empirical safety–utility results plus generalization evidence. SHE reports a 3.1× lower average ASR and 50.6% higher average utility under attack (UA) than the static SafeHarness baseline on Agent-SafetyBench, with transfer to held-out AgentHarm and to other agent models without additional evolution.

Main Findings

• Held-in improvement over the seed harness: On Agent-SafetyBench, SHE (evolved) reduces average ASR from 8.6% to 5.5%, lowers clean UBR (unsafe behavior rate on clean tasks) from 25.7% to 19.8%, and raises average UA from 33.5% to 47.6%, compared with SHE (seed).

• Improvement over static SafeHarness: SHE lowers average ASR from 17.1% (SafeHarness) to 5.5% and improves average UA from 31.6% to 47.6%, which the paper describes as a 3.1× ASR reduction and 50.6% higher UA.

• Lowest ASR and highest UA among compared methods: SHE (evolved) achieves the best average figures on these two metrics relative to all static and evolved baselines listed in Table 1.

• Transfer to unseen risks: On the held-out AgentHarm benchmark, the evolved harness reduces Harm Score from 19.8% to 9.8% relative to the non-evolved harness and raises Harm Refusal from 78.4% to 86.4%, while benign non-refusal (Benign NR) moves from 77.9 to 77.8.

• Cross-model transfer: A harness evolved on DeepSeek-V3.2 is applied directly, without additional evolution, to Kimi K2.6, GLM-5.2, and MiniMax M2.7, showing safety improvements over no defense with a comparable safety–utility trade-off across base agents.

• Functional decomposition matters: Under coupled designs such as LlamaFirewall and SafeHarness, evolution can reduce ASR but may cause regressions in clean safety behavior or utility under attack. SHE reports consistent improvement across all three reported metrics (clean UBR, ASR, UA).

• Learned components carry the gain: Replacing any single evolved artifact with its seed version (Rule Bank, Safety Memory, System Prompt, or Tool Policy) generally weakens the safety–utility trade-off, indicating the benefit comes from learned artifact-level refinements rather than the seed harness.

• Evolution model sensitivity: With GPT-5.5 as the evolution model, SHE converges at round R17 (clean UBR 19.8, ASR 5.5, UA 47.6). DeepSeek-V3.2 converges earlier at R03 (17.6, 4.3, 40.4) with a larger utility trade-off, and GLM-5.2 at R05 (24.0, 5.9, 36.0). For reference, no defense scores 42.9 clean UBR, 34.6 ASR, 38.7 UA in this table.

• Selection prevents regressions: The evolution history shows accepted updates at R00, R03, R04, R05, and R17, with a plateau from R05 to R16 during which candidates that improved only one metric were rejected for failing the joint safety–utility criterion.

Methodology in Plain English

The researchers fix the base model, the environment, and the evaluation protocol, and let only the harness change.

1. Fix the harness structure. The harness is written as four editable pieces — System Prompt, Rule Bank, Safety Memory, and Tool Policy. Each piece has a distinct safety job, which makes it possible to point at the piece responsible for a failure. The System Prompt and Safety Memory are textual specifications; the Rule Bank and Tool Policy attach safety conditions to intervention actions such as allow, warn, block, sanitize, and judge.

2. Roll out and diagnose. In each round, the current best harness is used to run a set of evolution tasks. Trajectories and outcome records are collected, safety-relevant cases are flagged, and each flagged case is turned into a structured diagnosis along three dimensions: harm domain (the potential consequence), attack surface (the channel through which risk enters), and failure mode (how the agent's behavior fails).

3. Route to the responsible artifact. Each diagnosis is assigned to the artifact — or small set of artifacts — that owns the relevant safety responsibility. This keeps edits local instead of rewriting the whole harness.

4. Make bounded edits. The evolution model proposes edits that specify target artifact, modification scope, update operation, learned content, and supporting trajectory evidence. Previously rejected edits and their reasons are fed back so later rounds avoid repeating them.

5. Check validity. Before evaluation, a validity check rejects edits that violate the artifact schema, add unsupported safety restrictions, remove needed capability, or produce only superficial safety gains.

6. Select the best harness. A candidate replaces the current best only if it improves safety and does not reduce utility. Otherwise the harness is restored and the rejection reason is fed back.

Safety Memory has a distinct update condition: it stores a boundary only when a failure remains unresolved after repeated attempts (specifically, unresolved after two rounds or recurring after artifact updates).

Setup specifics. Agent-SafetyBench contains 2,000 safety-critical tasks across 349 interaction environments covering 8 safety risk categories; the paper follows SafeHarness ordering and selects the first 200 tasks. A 15-task stratified subset (90 task–condition instances, each run with two rollout replications, so 180 trajectories per round) drives evolution, and the remaining 185 tasks are reserved for final evaluation. The full AgentHarm set (440 augmented harmful behaviors derived from 110 base tasks across 11 harm categories) is held out. Each task is evaluated under one clean condition and five attack conditions: context poisoning, indirect injection, tool tampering, memory injection, and composite attack. DeepSeek-V3.2 is the base agent; GPT-5.5 serves as the evolution model and the full-trajectory judge for Agent-SafetyBench, while GPT-4o judges AgentHarm. SHE runs 20 evolution rounds at temperature 0.

Why This Matters

Impact on research. The paper reframes agent safety as an architectural property that can evolve, not just a property of model weights or a fixed set of post-deployment guardrails. It also introduces responsibility attribution as an explicit design requirement for safety harnesses, arguing that coupled components make localized, regression-aware improvement difficult. This connects safety work to the line of research on optimizing agent artifacts from trajectory feedback.

Real-world applications:

• Tool-using assistant deployments where an agent books, purchases, installs, or modifies things on a user's behalf and must distinguish recommending an action from performing it.
• Enterprise agent platforms that need permission and execution boundaries around internal tools, plus a way to update those boundaries as new failure patterns appear.
• Runtime guardrail products such as prompt firewalls and lifecycle safety layers, which could adopt artifact-level decomposition to make their rules auditable and evolvable.
• Safety evaluation pipelines that want to convert existing multi-turn agent logs into targeted improvements rather than one-off reports.

Industry relevance. The reported cross-model transfer — evolving on DeepSeek-V3.2 and applying the harness unchanged to Kimi K2.6, GLM-5.2, and MiniMax M2.7 — suggests harness-level safety work need not be repeated per underlying model, which matters for teams supporting multiple model backends. The ablation showing that a harness leaked from benchmark-specific fixes is not the source of the gain (learned boundaries, not broad refusals) also matters for deployment quality, since over-refusal is a common commercial complaint.

Future Directions

• Scaling the evolution data. Evolution used only a 15-task stratified subset; whether larger or differently stratified subsets yield different boundaries, or faster convergence, is not established here.

• Attribution accuracy. The loop depends on correctly routing a failure to the responsible artifact. The paper does not report a quantitative measure of routing accuracy, leaving open how often attribution is wrong and what happens when it is.

• Cost and round budget. The reported history accepts updates at R00, R03, R04, R05, and R17 across 20 rounds with a long plateau; what a principled stopping rule would look like, and how much evaluation budget is needed, is not resolved.

• Generalization beyond the tested benchmarks. Transfer is shown to held-out AgentHarm and across three additional agent models. Transfer to entirely different domains (for example, robotics or code-execution agents), and to risks not represented in the diagnosis taxonomy, remains an open question.

• Interaction with the underlying model. SHE deliberately holds the base policy fixed; whether harness evolution and model-level safety training should be co-optimized is untouched.

Target Audience

Researchers and engineers working on LLM agent safety, agent harness or scaffolding design, and runtime guardrails will get the most from this paper. It is also relevant to practitioners deploying tool-using agents in production who need updateable permission and execution boundaries, and to evaluation researchers interested in converting multi-turn trajectory logs into targeted safety improvements. Readers looking for model-level alignment methods or adversarial attack generation will find it adjacent rather than central.

Authors’ abstract

The safety of large language model (LLM) agents depends not only on model weights but also on the agent harness that manages context, memory, tools, permissions, and runtime control. Existing safety mechanisms often treat the harness as a fixed deployment artifact, limiting their ability to evolve with emerging risks. Moreover, coupled functions across harness components obscure safety responsibility attribution, making localized evolution difficult. We propose Safety Harness Evolution (SHE), a framework that learns evolving safe boundaries from rollout trajectories. SHE decomposes the harness into four artifacts with explicit safety responsibilities, including the System Prompt, Rule Bank, Safety Memory, and Tool Policy, defining clear functional boundaries for localized evolution. Based on this decomposition, SHE introduces an attribution-guided evolution loop that converts trajectory failures into structured diagnoses, learns artifact-specific boundary refinements, and selects evolved harnesses through safety-utility validation. Experiments on Agent-SafetyBench demonstrate that SHE effectively enhances safety through harness evolution, achieving a 3.1x ASR reduction compared with static SafeHarness, while also improving benign utility. The evolved harness further generalizes to unseen risks on the held-out AgentHarm benchmark and transfers across agent models without additional evolution.