The Pulse
Experts Draft Incident Rules for AI Agents
A new arXiv paper draws on 23 academic and industry experts to propose what security reports should record when AI agents are compromised. The framework adds agent memory, autonomy, tool use, delegation and inter-agent communication to conv
AI.info Team ·
Existing AI incident reports often describe what an AI system produced. A new paper argues that reports about autonomous agents must also reconstruct how the system moved through memory, tools, delegated tasks and external services before harm occurred.
“Beyond Predictable Paths: Redefining AI Security Incident Reporting for Agents,” submitted to arXiv on September 21, 2026, synthesizes input from 23 experts in academia and industry. The authors say current reporting approaches do not adequately capture security incidents involving systems that pursue goals through multiple steps rather than returning a single output.
The proposal is not a finished standard. The paper is marked “under submission,” and its arXiv record says authorship does not imply endorsement of every subsection. Its value lies in identifying the evidence that investigators may need before an agent’s actions disappear from short-lived context or distributed system logs.
Why a Single Output No Longer Tells the Story
The paper separates conventional AI systems from modern agents built around large language models. An ordinary system may be investigated through an input-output pair; an agent may reach the same harmful result through a sequence of reasoning steps, tool calls, memory accesses and interactions with other systems.
That difference changes the object of investigation. A security report may need to explain not only what an agent did, but which data it read, what instructions entered its context, which tools it invoked, which identity authorized each call and whether another agent or subsystem influenced the result.
The authors describe agent security as a systems problem. A model may behave as designed while a vulnerability emerges from the interaction between the model, its memory, an orchestrator, a retrieval system, an external tool or a connected operating system. Under that model, a report focused only on the model’s output can miss the condition that made the incident possible.
The Proposed Record Follows the Agent’s Trajectory
The paper’s preliminary reporting scheme organizes evidence around the agent’s trajectory and capabilities. It calls for message histories, reasoning tokens, the provenance or version of external data, data reads and writes, and the difference between the autonomy allowed by design and the autonomy actually exercised during the incident.
That distinction matters because an agent’s theoretical permissions may exceed what it uses in practice. The paper says investigators should record the exact tools invoked, arguments passed and responses received, rather than relying on a list of tools that were merely available. Reports should also capture the agent’s effects on the underlying operating system, its identity during each action, authentication tokens and the full chain of delegation from a human or another agent.
Delegation can complicate responsibility. An agent might act with a user’s permissions, operate under a separate machine identity or pass work to another agent with different access. The proposed record therefore includes trust boundaries and the identities attached to individual tool calls, information that can help distinguish an improper access-control decision from malicious input, a compromised tool or a failure in communication.
Multi-Agent Systems Add a Second Layer of Evidence
The authors extend the reporting model to systems that contain sub-agents, retrieval models, classifier guards and judge models. The report would need to describe both the architecture defined before deployment and the components actually invoked during the incident.
Communication logs and orchestration protocols also become part of the security record. The paper points to failures involving weak message integrity, missing provenance, unsafe serialization, unclear authority and confusion between instructions and data. Malicious instructions can move between agents through ordinary communication channels, while coordination failures or collusion may arise only during execution.
Those incidents may not have a single defective component. A vulnerability can be an emergent property of the system’s interaction pattern, with each model or subsystem appearing to work as intended in isolation. Recording the sequence and relationships between components gives investigators a better chance of identifying that failure.
Reporting the Incident Can Create a New Attack Surface
More detailed evidence can improve reconstruction, but the paper warns that the reporting process can expose sensitive information. Runtime traces may contain proprietary data, personal information, credentials or confidential instructions. A system that requires extensive logging may therefore create a second problem: deciding what investigators must retain and what a regulator, customer or the public should actually receive.
The authors also identify attacks aimed at the reporting mechanism itself. A compromised agent could manipulate logs or alter access patterns to conceal activity. An attacker who understands the reporting rules might craft behavior that avoids detection, while a system used to file or analyze reports could itself disclose information or insert misleading content.
Those risks leave several design questions unresolved. The paper does not settle which fields should be mandatory, what evidence is sufficient for reproduction or legal review, how reports should protect privacy, or when incidents observed in one agent should be treated as evidence of a wider vulnerability.
Law Has Reporting Duties, but Not All the Fields
The paper places its proposal alongside existing regulation and standards rather than presenting a replacement for them. It discusses reporting duties under the European Union’s AI Act, including Article 73, and notes that current requirements do not specify fields for tool calls, memory writes or delegation in the level of detail the authors consider necessary.
The authors argue that logging decisions must happen before an incident. Agent memory can change, external context may be temporary, and a compromise may become visible only after a long sequence of actions. Waiting until the event is discovered may leave investigators without the evidence needed to reconstruct it.
The proposed framework is therefore a research agenda as much as a reporting template. Its immediate contribution is to move the unit of analysis from the model response to the agent’s complete operational path: what it knew, what it could do, what it actually did, who or what authorized each step, and how other components shaped the result.