The Pulse
BlackSwan Makes AI Agents Test Their Actions for Failure
A paper posted to arXiv on November 2 introduces BlackSwan, a method that has AI agents look for evidence against a proposed action before carrying it out. In simulated tasks, the method raises success rates on AppWorld and WebShop, while a
AI.info Team · Written with AI · fact-checked ·
“A wrong action often passes every check that asks whether it is right.”
Minhyeon Oh, Chiyeong Heo, Geonwoo Noh, Hoyoung Kim, Jangwon Son and Jungseul Ok, authors of “Action Verification through Conjecture and Refutation for Language Model Agents”
BlackSwan asks what could go wrong
The researchers posted their paper to arXiv on November 2, proposing a way for language-model agents to challenge their own planned actions before making changes in an environment. They call the method BlackSwan, after the idea that one counterexample can disprove a claim that many confirming observations cannot establish. The work targets agents that use tools to alter accounts, records or other external state based on instructions that may leave important details unstated.
Most action-checking approaches ask whether the conditions for a proposed action appear to be satisfied. BlackSwan instead asks what evidence would show the action is mistaken. The authors argue that a check built only to confirm an agent’s interpretation can miss facts that contradict it, such as a canceled dinner guest whose invitation still appears on the event page.
Three steps before a state change
BlackSwan applies an “expect, suspect, test” sequence to each proposed action that changes state. First, the agent states what it expects the action to accomplish; next, it writes down specific, testable ways that expectation might be wrong. Only then does it inspect existing observations or use additional read-only actions to test those possibilities.
The order matters: the agent must formulate its suspicions before gathering new evidence, so the evidence cannot shape the questions being tested. If a test confirms any suspicion, the system withholds the planned action and asks the agent to draft a replacement in light of what it found. If no suspicion is confirmed—or a question cannot be resolved without carrying out the action—the proposed change can proceed.
Higher completion rates on simulated tasks
The main evaluation uses AppWorld, a simulated environment with 457 APIs across nine applications. Its challenge test split contains 417 tasks, including requests that require agents to discover user-specific information and avoid unwanted or hard-to-reverse changes. The paper measures task goal completion by whether a task passes all its unit tests, eight on average and as many as 22.
On that challenge split, BlackSwan reaches an 84.7% task-goal completion rate when the underlying model’s thinking mode is on, and 81.5% when it is off. The authors report gains over the strongest baseline of 14.9 percentage points with thinking enabled and 25.2 points without it. On WebShop, a simulated online store, BlackSwan’s success rate is 50.8%, compared with 38.1% for the ReAct baseline, a 12.7-point difference.
The gains come with extra computation. Across three tested model backbones, BlackSwan completes 1.39 to 2.25 times as many tasks as ReAct, at 1.5 to 9.4 times the cost, with the largest relative gains on the weakest backbone. On the challenge tasks, the authors also count fewer errors: BlackSwan leaves 16.1 required changes undone per 100 tasks, compared with 24.9 for Accord, and reduces wrong values and collateral changes by 39% and 26%, respectively.
Verification cannot settle an ambiguous request
The results come from simulated benchmarks, not agents acting on real user accounts. The authors say pre-execution checks reduce errors but do not eliminate them, and they warn that real deployments need additional safeguards. A further limitation is ambiguity: if an instruction supports several plausible interpretations, BlackSwan may repeatedly surface the same unresolved concern and raise verification costs.
That trade-off puts the paper’s main claim in sharper focus. BlackSwan does not establish that an agent’s plan is correct; it changes the burden of checking by asking the agent to seek observable reasons not to execute it. In the AppWorld challenge split, the tested version completes 84.7% of tasks with thinking enabled, while the unresolved cases still require more than a verification pass.