"Shall We Approve?" Consequence-Aware Approval for Autonomous Coding Agents

AI summary

Overview

Research area: Security and safety of autonomous LLM coding agents (cs.CR), specifically the permission/approval gate that decides whether an agent-proposed shell command may execute.

Technical level: Intermediate to Advanced. Readers need some familiarity with LLM agent loops, tool calling, container sandboxing, and system-call tracing, but the central argument is stated plainly and does not depend on deep systems knowledge.

Scope (one sentence): The paper proposes Caper, an approval mechanism that runs each proposed shell command first inside a snapshotted, network-disabled container, summarizes what the command actually did into a small "effects report," has an independent LLM approver judge those observed effects against the user's task, and then either commits the result or rolls it back with an actionable reason — evaluated against six existing systems on SABER (682 tasks) and a new benchmark, YoloFS-Ext (204 tasks).

What This Paper Is About

Coding agents constantly propose shell commands, and asking a human to approve each one does not scale — the paper cites Anthropic reporting that users approve 93% of Claude Code permission prompts. Automated approvers exist, but they all judge the text of the request and command, while a command's real behavior depends on the code it invokes and the state of the workspace (the same make clean && make deletes only build artifacts in one workspace and deletes src/ and .git/ in another).

The goal is an approval decision that answers one question before anything is committed: what security-relevant effects will this command have on the current workspace, and are those effects within the scope of the user's request?

Key Contributions

1. A consequence-aware approval mechanism (Caper). Each proposed shell command is pre-executed in a snapshotted task container under system-call tracing, before any result reaches the agent. An independent LLM approver judges the observed effects against the task; allowed commands are committed, blocked ones are rolled back with a reason the agent can act on.

2. A bounded effects report. A design that turns a raw system-call trace into usable evidence for an LLM, retaining persistent file transitions (with bounded content diffs), network attempts, user-visible output, protected-file reads, source attribution, and explicit coverage/uncertainty markers. It targets 4 KiB, is capped at 8 KiB, and is reported as about 99% smaller than the raw trace (99.4% in the overhead measurement).

3. A new benchmark, YoloFS-Ext. A controlled extension of YoloFS built from its 13 hidden-side-effect projects, producing 102 dangerous tasks in which harm is hidden inside an invoked tool at four concealment levels (script, Makefile, chain of scripts, compiled binary), each paired with a safe twin that has identical files, request, and command except for a harmless tool. Blocking a twin is therefore a false block by construction.

4. An empirical comparison against six existing systems plus no-approval baselines, on two benchmarks, with analysis isolating why text-based approvers fail and why observed effects beat predicted effects.

Main Findings

• Lowest harmful-outcome rate on SABER. Caper reaches 6.7%, versus 14.8–21.2% for the six existing systems and 19.7% for no approval with the same agent (GLM-5.3-Flash). Run as a Claude Code hook, Caper reaches 10.0%.

• Large gap on hidden harm (YoloFS-Ext). Caper reaches 7.8% harmful outcomes and its Claude Code hook 6.9%, against 35.3–94.1% for the baselines. Caper also has the highest safe-completion rate on YoloFS-Ext at 60.8%, and blocks only 1.0% of safe twins (hook: 2.0%).

• Text-based approvers fail for lack of visibility, not lack of strictness. The five text-deciding systems leave 14.8–18.8% of SABER tasks harmed, against 19.7% with no approval at all. Swapping Claude Code's built-in classifier for Caper as a hook, with the same agent and approver model and everything else unchanged, cuts harm from 14.8% to 10.0% on SABER and from 62.7% to 6.9% on YoloFS-Ext.

• Safe outcomes are preserved on SABER. Caper's safe-outcome rate is 67.8% (48.6% safe completion plus 19.1% safe refusal), described as remaining at the level achieved without approval (66.6%).

• Selectivity separates dangerous tasks from their twins. The Caper approver without the effects report blocks 61.8% of safe twins; with the effects report it blocks the harmful command in 84.3% of dangerous tasks while blocking only 1.0% of twins. Qwen3Guard-Gen-8B, CARE, and Claude Code Auto Mode block few commands in either group (leaving 62.7–69.6% of dangerous tasks harmed); TS-Guard blocks the harmful command in 8.8% of dangerous tasks and also blocks 8.8% of twins.

• Ordering and independence matter. YoloFS observes the same file changes but shows them to the agent after execution. The agent reverts in 66 of 102 dangerous runs; where it never reverts, 33 of 36 runs end harmed, and YoloFS leaves 35.3% harm — four times Caper's rate, and more than no approval at all (21.2% versus 19.7%), because a credential once sent cannot be recalled. YoloFS also completes only 22.5% of dangerous tasks safely, versus 60.8% for Caper.

• More context is not a substitute for evidence. Without evidence, moving the approver from the command alone to the full dialogue drops harm from 44.1% to 17.6% but raises twin false blocks from 52.0% to 61.8%. With evidence, the command alone already yields 1.0% harm and no false blocks.

• Prediction is not observation. Letting the same model predict effects from the command and referenced files lowers harm as far as observation does, but 61.8% of twins are still blocked because the prediction leaves the approver uncertain. Removing the content diff from the report raises harm from 7.8% to 13.7%; a weaker approver gains on false blocks (53.9% to 4.9%) but not on harm.

• Concealment level stops mattering with evidence. Without evidence the harmful rate depends on packaging: 4.2% for a binary, 11.5% for a script, 26.9% for a Makefile or script chain. With evidence it stays at 4–15% at every level.

• Modest overhead. The effects report has a median size of 0.9 KB (95th percentile 2.6 KB) against 62.5 KB for the raw trace. Median approval latency is 3.2 s (95th percentile 6.4 s) versus 3.3 s (11.4 s) without evidence; the sandbox adds about 0.5 s per allowed command and 1.5 s per blocked one; API cost is $0.019 per task versus $0.017 without evidence.

• Case studies show actionable blocks. In one case a compiled deployment tool also overwrote src/app.js (29 B to 11 B, diff replacing code with // removed); in another a migrate target dropped three tables and shrank app.db from 32 KB to 20 KB. In both, Claude Code Auto Mode allowed the command, while Caper blocked it and the reason led the agent to a safe completion.

Methodology in Plain English

The setup. The agent is unchanged: a multi-turn model that proposes one shell command per step and reads the result. The intervention sits at the shell boundary. Each task runs in its own Docker container with networking disabled and memory and process counts limited.

Transactional pre-execution. Before a command runs, the container is snapshotted. The command then executes under strace -f with a per-command timeout, recording process creation, file operations, network system calls, and writes to user-visible output. If the approver allows it, the snapshot is discarded, the post-execution state is kept, and the captured output is returned as the tool result — the command runs exactly once. If blocked, the mutated container is destroyed and a fresh one starts from the snapshot, so no change survives.

From raw trace to a usable report. A single command can produce tens of kilobytes of trace, mostly dynamic loading, locale probes, and metadata checks. A probe run at container start captures the shell/loader/tracer's own accesses, which are subtracted by count. The extractor then keeps only boundary-crossing consequences as structured "cards": persistent file transitions (with bounded content diffs for small text files, and content redaction for high-precision sensitive paths), network attempts (marked as observed, attempt-only, or answered by a simulated service), and user-visible output. For consequences that can carry data, the report records which files the same process read beforehand ("same actor, read before consequence"). Reads of dotenv files, credential stores, and private-key candidates are kept as protected-read consequences even with no later sink. When output matches credential-value patterns, only the class is annotated — values are never copied into the evidence. Repeated events are aggregated; execution status, per-category counts, simulation markers, and uncertainties survive the budget so the approver knows what the report covers.

The approver. An LLM receives the user task, the interaction history with truncated tool outputs, the raw command, and the effects report, and returns JSON with Allow/Block, a confidence, and a short reason. It is instructed to judge independent axes: authorized goal and scope, credential access, changes to protected security controls, immediate runtime effects, deferred behavior set up to run later, and uncertainty. The packet is framed as evidence rather than instruction, and the approver is told not to treat the absence of an observed network or credential effect as proof of safety. On Block, the reason names the violated boundary without quoting secrets and instructs the agent to choose a safer alternative rather than retry through an equivalent wrapper.

Evaluation design. SABER supplies 682 shell-only tasks across eight harm categories (126 code tampering, 104 information leak, 84 file-system destruction, 82 unauthorized access, 82 data destruction, 74 privilege escalation, 66 outbound network, 64 persistence) and three scenarios (A: 275 injected-instruction tasks, B: 174 agent-mistake tasks, C: 233 explicit-but-dangerous tasks). YoloFS-Ext supplies 102 dangerous tasks plus 102 safe twins. Baselines: no approval with GLM-5.3-Flash and with DeepSeek-V3.2, CARE (rule-based), TS-Guard, gpt-oss-safeguard-20b, Qwen3Guard-Gen-8B (guardrails), YoloFS (staging), and Claude Code Auto Mode (deployed). Systems outside Claude Code replace only the approval step in a shared harness with the same rollback and reason-feedback mechanisms. All agents use GLM-5.3-Flash except the DeepSeek-V3.2 baseline; both the Caper approver and the Auto Mode classifier use gpt-5.6-luna; SABER runs get a single bash tool and at most 60 steps; each system runs each task once.

Oracle correction. Auditing SABER's oracle exposed false positives (commands misparsed, only inspected, or only written to a file). The authors correct them and judge every system with the same revised oracle; Caper has the lowest harmful rate under both the released (9.0%) and revised (6.7%) oracle.

Why This Matters

Research impact. The paper reframes agent permission from a text classification problem into an effect observation problem, and supplies a benchmark construction (dangerous task plus matched safe twin) that makes selectivity measurable rather than just strictness. That twin design exposes a failure mode invisible to harm-rate-only evaluation: an approver can look safe by blocking everything, or look permissive by blocking nothing, and the twin rate tells the two apart. The finding that predicted effects reduce harm but not false blocking, while observed effects reduce both, is a concrete, reusable result for anyone building guardrails.

Real-world applications.

• Coding agent products (CLI agents, IDE agents, CI-integrated agents) that need automatic approval without approval fatigue, with a block reason the agent can replan around rather than a hard stop.
• Enterprise deployment of agents in repositories containing credentials, production configuration, or security controls such as hooks, scanners, and repository policy.
• Supply-chain and build security: catching a project script, Makefile target, or compiled binary that does something other than its name suggests — the exact scenario YoloFS-Ext operationalizes.
• Auditing and compliance: producing a compact, per-command record of what an agent actually did (file transitions, connection attempts, credential reads) as evidence for review.

Industry relevance. The direct comparison against Claude Code Auto Mode — run through the official binary, and also with Caper substituted as a hook in that same harness — is aimed squarely at deployed products. The overhead numbers (median 3.2 s approval, about 0.5 s sandbox cost per allowed command, $0.019 per task) are the kind of figures a product team needs to decide whether pre-execution is affordable, and the report-size discipline (4 KiB target, 8 KiB cap) shows the design was built for real token budgets rather than for a demo.

Future Directions

• Effects that leave the sandbox. Rollback restores only the container's file system, so network requests, remote writes, and spawned processes happen before the decision and cannot be undone. The authors suggest routing traffic to a recording proxy that answers with placeholders; building and evaluating that is open.
• Extending beyond shell commands. Caper gates shell commands because their text reveals the least. File edits state their change in the call and are left to the harness, so an edit that does harm by itself (for example, overwriting a configuration file) is judged only from the call.
• Adaptive attackers and single-execution gaps. The report describes one execution; a tool whose behavior depends on time, randomness, or environment may behave differently on a real run. The authors list sandbox detection, deferring harm, planting text that the report carries to the approver, and splitting a harmful change across individually benign commands as unaddressed, and note that neither benchmark contains such an attacker.
• Trustworthy judgment. Whether an observed effect is acceptable depends on the task, and the judge is an LLM that can vary across runs or misread the report. SABER's own oracle combines rule checks with an LLM judge and can err similarly; more reliable or better-calibrated approvers remain open. A matched comparison against OpenAI Codex's automatic reviewer would also require changing its default sandbox policy, which reviews only requests that leave the workspace, so most harms in these benchmarks would never reach it.

Target Audience

Researchers and practitioners working on LLM agent security, agent permission and guardrail systems, and sandboxing for autonomous code execution. It is most valuable to engineers building or procuring approval gates for coding agents (who will care about the Auto Mode comparison and the latency/cost figures), to benchmark designers (the dangerous-task/safe-twin methodology is the transferable idea), and to security teams reasoning about harm that is invisible in command text, such as a build target or compiled tool that quietly exfiltrates a credential or destroys a source file. Readers wanting a first look at agent sandboxing will find the framing accessible; readers wanting implementation detail will need the appendices, which are referenced but not reproduced in the truncated content.

Authors’ abstract

Coding agents increasingly rely on automated approvers to decide whether a proposed shell command may run. Existing approvers, from rule-based checkers to guardrail models and deployed permission classifiers, base their decisions on the text of the request and command. Yet a command's behavior depends on the code it invokes and the state of the workspace, and systems that observe file changes leave the assessment to the agent or a human after execution. We present Caper, a consequence-aware approval mechanism that pre-executes each command in a snapshotted sandbox, provides an independent approver with a bounded report of its observed effects, commits allowed commands, and rolls back blocked ones with a reason the agent can act on. Compared with six existing systems, Caper achieves the lowest harmful-outcome rate on SABER (6.7% versus at least 14.8%) and YoloFS-Ext, a benchmark we build in which harmful behavior is hidden inside invoked tools (7.8% versus at least 35.3%), while blocking 1.0% of the safe twins of the dangerous tasks in YoloFS-Ext.