Risks and Controls for Multi-Agent Systems: an analytical framework for deployment of AI agents across organisational boundaries

AI summary

Overview

Research area: Governance, safety and risk management for multi-agent AI systems, particularly where agent-to-agent interactions cross organisational boundaries.

Technical level: Intermediate. The paper is an analytical/conceptual framework rather than a technical or empirical study, but it assumes familiarity with how AI agents are deployed and with basic concepts of organisational governance and standards.

Scope: A single report that proposes a framework for reasoning about interaction-driven risks between AI agents, how those risks shift as interactions cross organisational perimeters, and which actors are positioned to apply controls.

What This Paper Is About

As organisations deploy AI agents, those agents increasingly interact with each other — internally, with the agents of partners, customers and suppliers, and with unknown counterparties on the open internet. The report's core claim is that failures can emerge from the interactions themselves, and that once interactions cross an organisation's perimeter, no single organisation can fully see, control or govern them. Its goal is to give organisations, policymakers and researchers a shared way to reason about these risks and the controls that might address them.

Key Contributions

1. A three-tier taxonomy of agent deployment, defined by the minimum common governance binding any two interacting agents: singular governance (one organisation governs every agent), federated governance (multiple organisations deploy into a shared environment under agreed rules), and open environments (no central authority; shared standards are adopted voluntarily, if at all).
2. A per-tier analysis of risk factors, failure modes and available controls, so that risks are examined in the context of the governance arrangement that actually applies rather than treated generically.
3. An attribution of control responsibility, identifying which actors are positioned to apply which controls within each tier.
4. A characterisation of governance gaps: where no actor is positioned to act, the report describes the nature of the gap and the collective action that would be required to close it.

Main Findings

• Interaction, not just individual agents, is a source of failure. The report's starting premise is that failures can emerge from the interactions between agents themselves, a risk category distinct from the failure of any single agent.
• Organisational boundaries degrade visibility and control. Once agent interactions cross an organisation's perimeter, no single organisation can fully see, control or govern them — so the problem is structural, not merely a matter of better tooling within one firm.
• The applicable governance tier sets the ceiling on what controls are possible. The taxonomy is built on the minimum common governance binding any two interacting agents, which means the weakest shared governance arrangement between two parties determines the regime under which their interaction occurs.
• Singular governance is the most tractable case; open environments the least. Under singular governance a single organisation governs every agent; under federated governance multiple organisations operate under agreed rules; in open environments there is no central authority and shared standards depend on voluntary adoption. The report treats these as qualitatively different regimes rather than points on a continuum of the same problem.
• Some risks have no owner. The framework explicitly looks for situations where no actor is positioned to apply a needed control, and treats these as gaps requiring collective action rather than as oversights by a particular party.
• No quantitative results are reported in the abstract. The abstract describes the structure and claims of the framework but contains no empirical measurements, benchmark results, case studies or evaluation figures; any such material in the full report is not visible here.

Methodology in Plain English

The approach is analytical rather than experimental. The authors construct a conceptual framework by first asking what governance arrangement is minimally shared by any two agents that interact, and then using that question to sort deployments into three tiers. Within each tier, they work through the risk factors that arise, the ways those risks can fail, and the controls available. They then ask who is actually in a position to apply each control, and where nobody is, they describe the gap and what kind of collective action would be needed to close it. The result is a structured way of classifying a situation and reasoning about it, rather than a set of measured outcomes.

Why This Matters

Impact on research: The report reframes multi-agent risk as a governance and coordination problem rather than a purely technical one, giving researchers a vocabulary — deployment tiers, minimum common governance, unowned gaps — for comparing systems that operate under very different authority structures. It also points to a research agenda around settings where no single party can observe or control the interaction.

Real-world applications:

• Enterprise-to-enterprise agent collaboration, where a company's agents transact with a supplier's or partner's agents under a negotiated set of rules — the federated tier.
• Customer-facing and supply-chain agents, where an organisation's agents meet agents it does not select or vet in advance.
• Open-internet agent interactions, where agents encounter unknown counterparties with no central authority and only voluntary standards.
• Internal deployments, where a single organisation governs all interacting agents and can in principle apply controls directly.

Industry relevance: The framework is aimed at the practical question organisations face when deciding whether to let their agents interact beyond their own perimeter, and at what conditions. Because it identifies who is positioned to act in each case, it is directly usable by teams drafting deployment policies, by standards bodies deciding what to specify, and by policymakers assessing where voluntary arrangements are likely to be insufficient.

Future Directions

• Closing unowned gaps. The report identifies situations where no actor is positioned to apply a control; the obvious next question is what form collective action could take, who would convene it, and how it would be enforced.
• Turning the framework into operational guidance. The abstract presents tiers, risks and controls conceptually; translating them into checklists, contractual terms or deployment gates is left open.
• Making voluntary standards stick in open environments. Where adoption of shared standards is voluntary and possibly absent, how compliance is encouraged or verified remains unresolved.
• Validating the framework against real deployments. The abstract reports no empirical evaluation, so testing whether the tiers and failure modes match observed incidents and cross-boundary deployments is a natural next step.

Target Audience

Organisations deploying AI agents that interact with external parties; policymakers and regulators concerned with cross-boundary AI activity; standards bodies working on interoperability and agent governance; and researchers in multi-agent systems, AI safety and technology governance who need a structured way to describe where authority over an interaction actually resides. Readers looking for quantitative results or implementation detail will not find them in the abstract.

Authors’ abstract

This report presents a framework to help organisations, policymakers and researchers reason about the risks that emerge when AI agents interact with each other, how those risks change as interactions cross organisational boundaries, and the controls that may help address them. As organisations deploy AI agents, those agents will increasingly interact with each other: inside the organisation, with the agents of partners, customers and suppliers, and with unknown counterparties on the open internet. Failures can emerge from the interactions themselves, and once those interactions cross an organisation's perimeter, no single organisation can fully see, control or govern them. The report introduces three deployment tiers, defined by the minimum common governance binding any two interacting agents: singular governance, where one organisation governs every agent; federated governance, where multiple organisations deploy into a shared environment under agreed rules; and open environments, where agents operate with no central authority and shared standards are adopted voluntarily if at all. Within each tier, the report examines risk factors, failure modes and available controls. It identifies who is positioned to apply the controls, and where no actor is positioned to act, it characterises the gap and the collective action required to close it.