The Pulse
Trusted Cards Lifted AI Approvers’ Attack Catch Rate to 98%
A new arXiv paper tests how approval cards affect attacks on AI agents that can send messages, move money or change files. In trials with 15 language models as automated approvers, cards generated from the action itself raised the attack ca
AI.info Team · Written with AI ·
Approval cards that show what an AI agent is actually about to do raised automated approvers’ attack catch rate from 30% to 98% in experiments with 15 language models, according to a paper posted to arXiv on October 15. The result comes with a trade-off: the models refused 16% of harmless near-miss requests, mostly because the researchers used placeholder text in their test material.
The paper, “Approve What, Exactly? Effect Binding and Trusted Rendering for Agent Step-Up Authorization,” examines whether an approval really protects an action when an agent can influence what a person—or another model—sees. Its authors argue that an approval must be both beyond the agent’s control and tied to the precise effect that will execute. Otherwise, an agent might secure approval for one action and perform another, or mislead the approver about the action in the first place.
“Most deployed approvals fail one half of this criterion.”
Sai Varun Kodathala, Prashanth Pollishetty and Jaylen Cargill, authors of the paper
A card built from the action, not the agent’s story
The researchers built ASU, short for Agent Step-Up, as a reference design. A gateway outside the agent’s control converts a tool call into a structured description of its effect, then renders an approval card from that description. A passkey on the user’s phone signs a digest of the action, and the gateway executes only the stored call that matches it.
That design aims to close two gaps. A confirmation shown inside an agent-controlled chat may be answerable by the agent or an attacker who can write there. A separate approval screen can still mislead if the agent supplies the screen’s description; a card generated from the underlying call gives the approver a view independent of the agent’s narration.
Across 27,108 scripted trials and 609 AgentDojo attack pairs, ASU blocked tested substitutions, replays and attempts to stretch an approval into a broader grant. In a forged-narration test, where the agent described the requested action but asked to perform a different one, attack success fell from 54% with an agent-written description to 3% with the trusted card and an exact-comparison approver.
Runtime approvals leave gaps
The paper also audits ten agent runtimes. Of the eight where the authors could measure whether approval stayed bound to the action, four allowed a resumer, hook or stored history to change what ran after approval. The authors say none of the ten produced an approval record that a third party could verify using the user’s public key.
Flooding remains a separate weakness. In the seven runtimes where the researchers could measure limits, none restricted approval floods within a run. In their ASU experiments, rate limits and denial lockouts reduced the modeled approver’s chance of eventually accepting a malicious request, but did not eliminate it: flooding succeeded 45% to 60% of the time under the study’s assumptions.
The runtime findings describe the approval mechanisms examined, not every possible configuration: the authors say integrators can close some gaps by rendering structured arguments carefully and resuming runs only through a trusted service. The paper also tests specific designs, including Google’s Agent Payments Protocol, Microsoft’s action-bound approval design and Model Context Protocol elicitation. Its results do not establish that every implementation of those systems has the same weaknesses.
The 98% result is not a human study
The 98% catch rate measures language models acting as automated approvers, not people. The paper reports no human-subjects data; its human approvers are simulated with models and decision rules. The finding therefore shows how the tested models reacted to different displays, not how reliably human users would spot a deceptive request.
The false refusals also need context. Of 103 benign cards refused in the language-model tests, 88 refusals cited placeholder text—such as a message body marked “report placeholder”—that appeared in the card but not in the agent’s narration. The authors calculate that the false-denial rate would have been 2% without those placeholder-related refusals, while cautioning that real deployments need to display content in a form users expect.
A second device produced no measurable change in the experiments compared with a trusted confirmation on the agent’s own host. The authors say that result reflects the tested conditions: the experiments did not compromise the host’s display, while the phone provides a separate trust domain and a signature that others can verify. The paper leaves the practical human question unanswered: whether people will notice the same mismatches when real approval cards, real consequences and repeated prompts replace scripted tests.