From Evidence to Action: How Tool-Using Agents Fail

AI summary

Overview

• Research area: Evaluation and safety of Large Language Model (LLM) agents that use tools to change external state (customer service, infrastructure, finance, smart-home and healthcare-style operations).
• Technical level: Advanced. It assumes familiarity with agent harnesses, tool-calling, trajectory evaluation, bootstrap confidence intervals, and dependency graphs.
• Scope: The paper introduces SafeActBench, a 656-case benchmark across six operational domains and five protocols, and uses a provenance-bound Evidence Ledger plus a deterministic trajectory evaluator to localize where the chain from "evidence established" to "consequential action executed" breaks across ten model–harness configurations.

What This Paper Is About

Tool-using agents can produce the right outcome — a correct refund, a correct record update — even when the agent never established, in its observable trajectory, that the action was justified. This paper asks whether each consequential action was actually supported by evidence gathered beforehand, and where that evidence-to-action chain breaks as tasks move from a static decision, to justified inaction, to single actions, and then to dependent multi-action workflows. The authors build a benchmark (SafeActBench) to measure this directly rather than treating task success as a single yes/no outcome.

Key Contributions

1. A formulation of consequential tool use as an observable evidence-to-action chain linking investigation before action, execution, and downstream dependencies between actions — evaluated from observable trajectories rather than hidden reasoning.
2. SafeActBench, a benchmark suite of 656 cases across six operational domains and five protocols, spanning a static decision setting (Legacy) and four interactive regimes (V0–V3), from investigated non-action up to dependency-constrained multi-action workflows.
3. Evaluation machinery: a provenance-bound Evidence Ledger that binds established facts to the source interactions and entities/states they describe, and a deterministic evaluator that replays trajectories and scores required investigation, exact tools/targets/arguments, execution receipts, action dependencies, and resulting state changes without an LLM judge.
4. A characterization of where the chain breaks across ten model–harness configurations, plus controlled evidence interventions showing that agents may retrieve relevant information yet still act when decisive support is absent.

Main Findings

• Static judgment is strong; interactive execution is much weaker. Across ten model–harness configurations, Legacy (static Allow / Block / Defer) scores range from 77.9% to 97.7%, while overall exact case success (ECS) ranges from 37.7% to 67.2%. GLM–ZCode and DeepSeek–DSH both exceed 96% on Legacy, yet DeepSeek–DSH stays around 60% on V1–V3 while GLM–ZCode falls to between 12.1% and 34.1%.
• The gap persists when case identity is fixed. For three configurations re-evaluated on the same V1 cases, static accuracy is at least 95% while interactive ECS is at most 52% (Appendix C.2).
• Most failures start before execution. Stopping before required investigation (BSR) on V0 ranges from 21.7% to 62.9%, and acting before evidence completion (PAR) ranges from 37.0% to 66.9% among V1 episodes with an action attempt.
• Once evidence is established, single-action execution is usually reliable. Conditional on an action attempt after evidence completion (CAS), nine of ten configurations score between 93.2% and 100%; DeepSeek–Inspect is lower at 83.3%.
• Multi-step workflows show several distinct failure signatures. V2 and V3 additionally expose unresolved prerequisites at an action checkpoint (Gap) and executed-but-incomplete workflows (Part.). GLM–ZCode has a far higher Gap than Part. rate on V2 (80.3% vs. 33.3%), whereas the two rates are comparable for Qwen–Inspect (27.3% vs. 29.5%). The authors note Gap and Part. can co-occur, so they are overlapping diagnostics, not an additive decomposition.
• Harness effects are model-dependent. On the same 570 V0–V3 cases with 95% case-paired bootstrap CIs (5,000 resamples), DeepSeek gains +4.4 points with DSH over Inspect (CI +1.1 to +7.9) and GLM shifts −6.8 points with ZCode versus Inspect (CI −11.2 to −2.5); both intervals exclude zero. Claude (+4.2, CI −0.7 to +9.1), GPT-5.6 (+4.7, CI 0.0 to +9.5) and Qwen3.8 (+2.8, CI −1.2 to +6.8) intervals include or reach zero.
• Aggregate scores mask case-level disagreement. Treating a case as successful under a harness if at least two of three rollouts succeed, the two Qwen harnesses disagree on 134 of 570 cases (23.5%): 75 succeed only under Qwen Code and 59 only under Inspect.
• Harness changes often shift the type of failure rather than uniformly reducing it. On V1, GLM under Inspect rather than ZCode trades early-action failures (0.66 to 0.29) for no-action failures (0.01 to 0.37), leaving overall failure rate nearly unchanged.
• Withholding evidence reduces but does not eliminate action. In controlled interventions on 43 V1 cases (DeepSeek–DSH, GLM–ZCode, Qwen–Qwen Code), withholding a decisive record reduces action probability by 37.2–45.2 percentage points, yet agents still act in 46.5–53.5% of completed Withheld episodes. Contradiction reduces action probability by 54.5–70.0 points on its eligible subset of 22 cases. Among the 66 Withheld episodes with an action, 65 include a call to the affected tool (though this does not establish the absence was observed before acting).
• Agents respond more to a requester's claim than to absent evidence. With the same 43 completed cases per configuration, the evidence-package condition reduces action submission from 23/43 to 18/43 (DeepSeek), 20/43 to 14/43 (GLM) and 23/43 to 13/43 (Qwen); only Qwen's paired interval excludes zero. Requester-claim reduces action submission to 6/43, 10/43 and 6/43 respectively, with all three intervals excluding zero.
• Retrieval effort and action restraint do not move together. Urgency cues and explicit system warnings about missing evidence have paired intervals that include or reach zero for all three configurations. Under withholding, distractors increase mean information calls by 1.0–1.2 per episode (all three intervals above zero) while the corresponding action-probability changes all have intervals including zero.
• The benchmark is validated against itself. All 656/656 author reference solutions pass the deterministic evaluator, all 277 shared-world cases can be reconstructed, 2,468 frozen-evidence deltas pass schema validation, and all 132 V3 workflow DAGs pass structural checks.

Methodology in Plain English

The authors separate tool calls into information-gathering calls and consequential actions that change task-relevant state. For each consequential action they define a set of requirements — facts about the target entity, current state, amount, authorization, version, or other conditions — that must be established before execution. A requirement counts as satisfied only when the task context or a prior observation supplies valid evidence about the right entity at the right time. Reading an amount from one charge, for example, does not establish the amount of a different charge even if the numbers match.

Building on that idea, SafeActBench uses five protocols. Legacy requires a structured Allow / Block / Defer judgment on a fixed candidate action. V0 requires the investigation needed to justify not acting. V1 requires all necessary evidence before exactly one correct action, after which the episode ends. V2 extends this to a linear workflow where later actions may consume actual earlier results, and V3 permits any valid topological order consistent with a directed acyclic dependency graph. Requirements and expected actions are hidden from the agent, which sees only the task instruction and tool interfaces.

Evaluation is done by replaying recorded trajectories — tool interactions, action receipts, terminal outputs, resulting state changes — against the private case specification, with an Evidence Ledger tracking provenance for each fact. Scoring is deterministic and does not use an LLM judge or inspect hidden reasoning. The primary metric, exact case success (ECS), is binary per episode; the authors also report unweighted protocol-macro (P-M) and domain-macro (D-M) averages. Each case–configuration pair is run three times, rates are computed within each repetition and averaged with equal weight, and the main evaluation had no infrastructure failures, giving 100% coverage.

To understand why agents fail, the authors derive conditional diagnostics (stopped before required investigation, acted before evidence completion, success conditional on action after evidence completion, unresolved requirement at a checkpoint, incomplete workflow) and run paired harness comparisons with case-paired bootstrap confidence intervals. They then run controlled interventions on a subset of V1 cases — withholding a decisive record, contradicting a prerequisite field, presenting the tool responses directly, having the requester claim the missing record was already checked, adding urgency cues, adding explicit missing-evidence warnings, and injecting irrelevant distractor records — while keeping the public task and tool interface fixed.

Why This Matters

The paper argues that reaching the intended final state is not sufficient evidence that an agent behaved correctly: an action can be permitted by the full task state, use the correct tool, target and arguments, succeed, and still be unsupported by anything the agent established beforehand. For systems that make irreversible or externally visible changes, that distinction matters, because missing support can propagate to downstream actions.

Impact on research. The work shifts evaluation from task-completion or function-call accuracy toward action-specific, provenance-aware support, and positions SafeActBench against benchmarks such as AgentBench, WorkArena, OSWorld, WorkBench, AppWorld, τ-bench, ToolSandbox, Agent-SafetyBench, Near-Miss, EnvTrustBench and AgentAbstain. Its comparison table claims SafeActBench is the only listed entry with explicit criteria for pre-action evidence, entity/state binding, result propagation, and investigated non-action simultaneously. It also connects to the near-miss observation that agents can bypass policy checks yet reach the correct final state.

Real-world applications (the paper's six synthetic domains, which the authors state should not be read as safety certifications for the corresponding real deployments):

• Customer and policy operations, such as refunding a duplicate charge and notifying the customer, where the refund conditions must be verified for the specific charge.
• Engineering and infrastructure operations, where agents update records or trigger downstream operations.
• Legal and financial operations, where authorization, versions and amounts must be established before a state change.
• Smart-home control and healthcare operations, where actions must respect dependencies and confirmed resulting states.

Industry relevance. The harness comparisons matter for teams that choose an agent scaffold: harness choice changed success by +4.4 points (DeepSeek, DSH over Inspect) and −6.8 points (GLM, ZCode versus Inspect), and in some cases merely relocated the failure mode. The result that agents sometimes act even after querying the affected tool, and respond more to a requester's assurance than to missing evidence, points to concrete deployment risks around confirmation and social pressure.

Future Directions

1. Extend the controlled interventions to multi-action protocols, varying evidence at each action checkpoint, to connect observable support with what the agent internally relied on — the authors explicitly call this a natural next step.
2. Close the gap between observable support and internal reliance. The paper states its analysis captures observable support rather than what the model actually relied on, leaving that link open.
3. Investigate why harnesses change failure types rather than failure rates, including the case-level disagreement observed between harnesses on the same cases (134 of 570 for the two Qwen harnesses).
4. Determine why agents keep acting when decisive support is absent, given that distractors increase information calls without reducing action probability, and that requester claims reduce action more than the evidence itself does.

Target Audience

Researchers and engineers working on agent evaluation, tool-use safety, and benchmark design; practitioners selecting model–harness combinations for agents that make consequential changes; and policy or trust-and-safety teams who need to distinguish "the task completed" from "the action was justified." It is most useful to readers already comfortable with agent harnesses and trajectory-based evaluation.

Authors’ abstract

Tool-using agents make consequential changes to external state, yet correct outcomes do not guarantee that their actions were supported by evidence established beforehand. We study where this evidence-to-action chain breaks as agents move from deciding whether to act to executing single actions and dependent workflows. Across ten model-harness configurations, strong static action assessment can coexist with much weaker interactive execution. Failures often begin before execution: agents stop with incomplete investigation or act before required evidence is established. Once required evidence is obtained, single-action execution is usually reliable, while multi-action workflows additionally expose unresolved prerequisites and incomplete execution. For this analysis, we introduce SafeActBench, comprising 656 cases across six operational domains and five protocols that progress from static action judgment and investigated non-action to single- and multi-action workflows. A provenance-bound Evidence Ledger and deterministic trajectory evaluator track what information was established, when actions occurred, and whether downstream dependencies were satisfied. These results show that failures arise not only from missing information, but also from how agents use established evidence when deciding and executing actions.