RelLedger: Relational Information Governance for LLM Agent Organizations

AI summary

Overview

Research area: Multi-agent systems — specifically, auditing and governance of how large language model (LLM) agents decide which facts enter a shared organizational record, and what downstream consequences those record-keeping choices have.

Technical level: Advanced. The paper combines an experimental audit design (crossed actor histories, role reassignment, history rewrites, record controls) with formal identification arguments (a proposition and corollary on intervention coverage and finite-sample confidence regions), so readers benefit from familiarity with causal-inference and experimental-design vocabulary. The core question, however, is stated plainly and the results are reported in percentage-point contrasts.

Scope: The paper introduces RelLedger, a framework that holds an event, its message, its risk estimate, and its task conditions fixed while varying the relationship history associated with different actors, in order to measure whether relational context changes whether the same fixed event report is written to a shared organizational record and reaches later decisions.

What This Paper Is About

In an organization of LLM agents, an agent handling an event acts as a gatekeeper for organizational memory: it decides whether a fact gets written into the shared record that other agents later read. The problem this paper addresses is that when a sender's history with a counterparty influences that write decision, a "delivery gap" alone does not reveal whose relationship drove the choice, whether the effect follows a person or a role label, or whether the resulting record difference matters to anyone downstream. RelLedger is the proposed audit: it keeps the event and its message fixed, crosses sender histories across actors, and traces the effect from the write decision through the shared state into a recipient's later decision.

Key Contributions

1. Identifying "relational policy leakage." The paper names and measures the phenomenon where relational context changes whether the same fixed event report enters a shared organizational record and reaches downstream decisions. The authors report that the largest effects arise in discretionary delivery tasks, while explicit loss-rule tasks show small relation gaps.

2. The RelLedger audit framework. RelLedger uses crossed histories, role reassignment, history rewrites, and record controls to locate which cue governs delivery. Its attribution ledger maps each claim to the required response cells and separates missing interventions (coverage) from sampling uncertainty.

3. A large-scale controlled audit across five models. The evaluation covers GPT-5.6-Sol, Gemini-3.7-Flash, Claude-Sonnet-5, DeepSeek-V4-Flash, and Qwen-3.8-Flash across 302,764 valid records, revealing model-specific control paths and their task conditions.

4. Executable workflow validation. The audit is extended to 24 executable "one-person company" (OPC) workflows across eight operational modules, tracing retrieved organizational histories through tool calls, shared ledger updates, and recipient decisions.

Main Findings

• Claude shows the largest factual-notice delivery gap. Claude sends the same factual notice 62.22 percentage points more often with conflict histories than with friendly histories. The effect persists across robustness checks: a context-clustered bootstrap gives a 95% interval of [54.44, 70.00], leave-one-context-out estimates range from 60.61 to 64.24 points, and leave-one-template-out estimates range from 59.03 to 68.06 points.

• Message meaning and interface change the observed pattern. For Claude, the conflict–friend delivery gap is 62.22 points for a factual notice but 1.11 points for a collaboration warning. GPT has an equivalent −1.67-point notice contrast; Qwen has an equivalent 3.89-point notice contrast and a 17.22-point warning contrast.

• Different models follow different control paths. Gemini's −20.00-point actor swap effect follows the event person. Claude shows directional target and unrelated actor effects of 28.61 and 19.17 points. GPT has equivalent benchmark-average base actor effects, while DeepSeek and Qwen retain open base effects. In role reassignment, Gemini has an equivalent −0.23-point label effect, whereas Claude, DeepSeek, and Qwen show directional label effects of 21.76, 26.62, and 26.62 points; GPT's 14.12-point label effect remains open.

• History rewrites preserve the direction of some profiles but not the magnitude of all. Under independent wording, Gemini's actor swap remains negative and DeepSeek's remains near zero, while Claude's target effect ranges from 9.44 to 50.56 points and Qwen retains a positive role-label effect.

• Explicit loss rules produce small relation gaps. Across 54,000 fixed-message decisions spanning five risks and two cost pairs, every conflict–friend cell stays below at least one pre-specified detection criterion. Gemini's conflict–friend threshold gaps stay below 0.47 points, and a shared-boundary rule reaches 0.186-point mean absolute error on the held-out cost scale, compared with 0.489 points for a relation-dependent rule.

• Record differences propagate to recipients. The Claude sender delivers 81.67% of conflict records and 15.00% of friend records. Recipients see only those records, and normal-record checking gaps across five recipient models span 53.33–66.67 points. Forced release reduces the largest absolute gap to 13.33 points, and full information reduces it to 11.67 points.

• Executable workflows reproduce the pathway. For Claude, strained histories reduce individual sender writes by 37.50 points. In the majority-aggregated workflow, record presence changes from 87.50% under warm histories to 37.50% under strained histories, and recipients show a 45.83-point review reduction with a corresponding 0.229 increase in expected loss. The sender effect is negative in seven of eight modules and remains −33.33 points under the independent rewrite.

• Intervention coverage bounds what an audit can claim. In known-policy validation across 43,200 audits with 24 estimands each, target-only queries point-identify 12.5% of estimands, a crossed slice raises this to 33.3%, and the actor-swap design reaches 54.2%. The complete 3×3 grid point-identifies all 24 estimands. With 3,240 calls, correct-claim coverage rises from 10.4% to 36.6% across these designs, and the known-policy audits produce no false familywise declaration.

• Model competence controls generally passed, with exceptions. Reliable evidence increased reported belief by 23.9–24.2 points over less reliable evidence. On supplied-prior numerical controls, GPT, Gemini, and Claude reproduce the supplied prior with mean absolute errors of 0.10–0.16 points; DeepSeek reaches 1.81 points of posterior error and follows direct sample calibration on 75% of skewed-count responses, and Qwen's posterior error is 5.82 points. In the free-message arm, message fidelity ranges from 94.32% to 99.02%, and retention of the stated risk ranges from 67.44% for Claude to 100% for Gemini.

Methodology in Plain English

RelLedger treats each organizational record update as a single "delivery transaction" with a sender, an event target, an unrelated actor, and a recipient. The researchers hold the event, the message text, the independently verified risk estimate, the recipient, and the decision conditions fixed. They then vary the sender's history with the actors — labelled friend, stranger, or conflict — and observe whether the fixed message gets written to the shared record.

Because a prompt containing two histories creates ambiguity about which relationship matters, the design crosses them: the target's history and an unrelated actor's history are varied independently, producing a 3×3 surface. This lets the authors compute a target effect (does delivery depend on the target's history, holding the other actor fixed), an unrelated-actor effect, an actor-swap contrast, and an interaction that tests whether target dependence changes with the other person's history.

To check whether an effect follows a person or a role, the researchers reassign the event role — assigning the event to the target or to the unrelated actor — while counterbalancing history assignments. This separates an average role contrast from history-swap modulation. They also repeat the histories with independent wording to test whether the pattern survives rewording.

To test whether the effect is about task discretion rather than about relationships per se, they vary message meaning (factual notice versus collaboration warning), the response interface (rating versus executed delivery), and stated costs. Under explicit loss rules, the optimal risk threshold is defined by the ratio of false-positive cost to total cost, and they check whether send rates track that threshold rather than the relationship.

Finally, they run recipient replay: a separate recipient model sees only the resulting record (the relation text is hidden) and makes a decision. They also implement 24 executable one-person-company workflows across eight operational modules, where a sender reads stored CRM history, handles a monitoring report with a fixed risk estimate, and calls a tool that either updates a JSON ledger or completes the task without a write. Three sender runs determine the average tool-write rate and one majority record; recipients then read that state under normal, forced-write, and full-information arms.

Statistically, the framework uses partial identification: a proposition shows that an attribution claim is point identified exactly when every response cell it requires has been queried, and a corollary extends this to a finite-sample outer confidence region. Percentage-point effects receive directional, equivalent, or open claims at a ten-point threshold; expected-loss effects retain their native units. Empirical intervals use 10,000 paired scenario bootstrap samples, and the headline notice effect receives context-clustered and leave-one-group-out checks. Scenario construction used 120 English scenarios drawn from 24 organizational contexts and five event templates, with the delivery grid crossing five risks, two cost pairs, three relations, and three independent repeats.

Why This Matters

Impact on research. Prior work on long-term memory, sycophancy, and social reasoning shows that relational and social context can change what a model says or judges. This paper shifts the question to what a model records: whether an agent's relationship history determines which facts enter a shared organizational memory. It also contributes a methodological point beyond the specific result — a large measured gap is not the same as an identified actor effect, and the attribution ledger makes that distinction explicit by reporting which response cells an attribution claim requires.

Real-world applications.

• Enterprise customer-support and CRM agents: deployed agents already carry persistent customer information across conversations and channels, and are grounded in account histories, product usage, service logs, and prior interactions. An audit like this tests whether a customer's relationship history changes what gets written into the account record.
• Case management and record-updating agents: where agents update customer records from conversations and emails, RelLedger's record controls indicate whether forced writes or fuller information narrow downstream gaps.
• Security and risk monitoring workflows: the paper's worked example is a security-monitoring case in which an unusual login was dismissed without an account review entry; warm histories produced writes on 3/3 runs and 2/3 recipient reviews, while strained histories produced 1/3 writes, an empty majority record, and 0/3 reviews.
• Multi-agent tool and ledger pipelines: the executable OPC workflows connect retrieved organizational histories to tool calls, shared JSON ledger updates, and recipient decisions, which is the shape of many operational agent deployments.

Industry relevance. The paper's setting is one in which a human owner cannot inspect every interaction or record update, so the agent handling the event serves as a gatekeeper for organizational memory. When relational context affects whether a fact is recorded, it shapes what information becomes available to the whole organization. The authors argue that the record-write boundary is a concrete and auditable point for governing organizational visibility, and that the intervention studies quantify how much of a downstream difference is removed when the record is controlled.

Future Directions

• Broadening intervention coverage to identify more claims. The known-policy validation shows that target-only queries identify only 12.5% of the 24 estimands and that the full 3×3 grid identifies all of them. The natural open question is which additional interventions are needed to point-identify interaction and history-modulation contrasts, which remained open under familywise intervals in the model experiments.

• Understanding why explicit loss rules suppress relation effects. Relation gaps stay small when costs and thresholds are explicit, while the largest effects appear in discretionary delivery. What makes a task "discretionary" versus rule-governed, and whether task specification can be used as a de-biasing lever, is left open.

• Connecting relational governance to existing safety and privacy work. The related work points to agent communication safety, contextual integrity in enterprise workflows, source preference, and shared-memory trust research. The paper does not report experiments that integrate RelLedger's delivery audit with those privacy and provenance concerns.

• Testing mediation pathways against alternatives. The paper reports that record controls sharply reduce the downstream gap and that forced release and full information reduce recipient-checking gaps, but it does not identify the internal mechanism by which relational context enters the sender's write decision, nor whether the same profiles generalize beyond the five evaluated models and the 24 synthetic organizational contexts.

Target Audience

This paper is most useful to researchers and practitioners working on multi-agent systems, LLM agents deployed in enterprise settings, and AI auditing or governance. It will be particularly valuable to (1) safety and alignment researchers studying how social and relational context steers model behavior beyond text generation, (2) evaluation and benchmark designers who need designs that attribute an effect to the correct cause rather than to a prompt-level correlation, (3) engineers building agentic workflows with shared memory, tool calls, and record updates, and (4) enterprise governance and compliance teams concerned with what information an automated agent writes into an organizational system of record. Readers without a causal-inference background can follow the experimental contrasts and the reported percentage-point effects, while the identification and coverage arguments in Sections 2.2 and Appendices A and E are aimed at readers comfortable with partial identification and simultaneous confidence intervals.

Authors’ abstract

In LLM agent organizations, shared records determine which facts are available for later decisions. A sender consults interaction histories before deciding whether to report an event. A delivery gap across histories does not show whose relationship drove the choice or whether the resulting record matters downstream. We introduce RelLedger to audit these links. It holds the event, evidence, risk, and message fixed while crossing the sender's histories with the event target and an unrelated actor. Role reassignment and history rewrites locate the controlling cue. Message and cost interventions show when the effect appears. An attribution ledger maps each claim to its required response cells and separates missing interventions from sampling uncertainty. Recipient replay follows delivered information into downstream decisions. Executable workflows for a simulated one-person company extend the audit to retrieved histories, tool calls, and shared ledger updates. Across GPT, Gemini, Claude, DeepSeek, and Qwen, the audit reveals model-specific control paths and smaller relation gaps under explicit loss rules. Claude sends the same factual notice 62.22 percentage points more often with conflict histories than with friendly histories. In executable workflows, strained histories reduce recipient review rates by 45.83 points relative to warm histories for Claude.