MPCI-Bench: A Benchmark for Multimodal Pairwise Contextual Integrity Evaluation of Language Model Agents

AI summary

Overview

Research area: Privacy evaluation for multimodal language-model agents, specifically benchmarks grounded in Contextual Integrity (CI) theory.

Technical level: Intermediate. The paper assumes familiarity with LLM/MLLM evaluation, agent tool-use sandboxes, and privacy-norm frameworks, but its core argument is accessible to anyone working on AI safety or data governance.

Scope: The paper introduces MPCI-Bench, a 2,052-case, 10-domain benchmark of paired positive/negative multimodal privacy cases spanning three tiers (Seed, Story, Trace), and reports evaluation results for ten frontier and open-source multimodal models.

What This Paper Is About

Existing Contextual Integrity benchmarks for language models are largely text-only and mostly test whether a model can refuse inappropriate requests. That leaves two gaps: they do not measure privacy risk that arises from images, mixed-media documents, or screenshots, and they do not measure the flip side, namely whether an agent still shares information when sharing is appropriate and needed to complete a task.

The authors build MPCI-Bench to test both sides of this privacy–utility trade-off, at both the level of normative judgment and the level of executable agent action.

Key Contributions

1. MPCI-Bench, described as the first benchmark (to the authors' knowledge) that evaluates contextual integrity in multimodal, agent-based settings, with 2,052 cases across 10 contextual domains.
2. A pairwise, multi-tier evaluation framework — Seed judgments, context-rich Story reasoning, and executable agent action Traces — where each positive and negative instance is derived from the same source image, so that appropriate and inappropriate information-sharing decisions are measured against matched visual content.
3. A Tri-Principle Iterative Refinement (TPIR) construction pipeline that uses an LLM-as-a-judge to enforce Semantic Utility (P1), Contextual Appropriateness (P2), and Story Realism (P3), validated by a human study.
4. A comprehensive evaluation of frontier MLLMs, revealing systematic privacy failures including "utility-biased oversharing" and a pronounced modality leakage gap favoring visual leakage over textual leakage.

Main Findings

• Models probe well on norms in isolation. At the Seed tier most MLLMs reach high F1; InternVL3.5-8B is highest at both the Seed tier (accuracy 0.978) and the Story tier (accuracy 0.951), outperforming larger models such as GPT-4o. GPT-5 reaches a Seed F1 of 0.973.
• Accuracy does not scale monotonically with model size. Within the Qwen3-VL family, Qwen3-VL-30B performs substantially worse than Qwen3-VL-8B at the Seed tier (0.829 vs. 0.953 accuracy).
• The Story tier exposes divergent privacy–utility strategies. Introducing a task-driven reason to share the image degrades performance. GPT-4o adopts a conservative stance (Recall 0.699, Precision 0.968), while InternVL3.5-14B leans toward oversharing (Recall 0.953, Precision 0.858). Only a subset, such as GPT-5 and InternVL3.5-8B, keep Story-tier F1 above 0.93.
• A large modality leakage gap appears in agent actions. On negative cases, visual leakage is consistently higher than textual leakage. In Table 4, GPT-4o shows a textual leakage rate of 40.9 versus a visual leakage rate of 90.4. (The paper's narrative text cites 43.7% textual and 90.4% visual for GPT-4o, and describes GPT-5 as having the lowest aggregate leakage rate at 76.8%, whereas Table 4 reports 60.2 for GPT-5; the visual figure of 56.9 for GPT-5 is consistent across both.)
• The dominant leakage mechanism is blunt file attachment. On average, 59% of image leakage in negative cases occurs through attachment-only actions, with a smaller share involving semantic description or a combination of both.
• Utility on positive cases is high. In positive scenarios, where sharing the image is the expected behavior, all evaluated models exceed 90% visual utility.
• A probing–action gap. Models that recognize privacy norms correctly in a binary probe still leak: the paper notes that Qwen3-VL-30B attains high trace-tier probing F1 yet still leaks images in over 90% of negative traces.
• Prompt-based mitigation trades privacy against utility. For Mistral-Large-3, Chain-of-Thought drops the visual leakage rate from 82.1% to 8.6% but also collapses utility to 13.4%. The CI Filter prompt is described as the most balanced: it lowers Mistral's leakage to 29.6% while keeping utility at 92.0%, and lowers Qwen3-VL-8B's leakage to 13.1% while keeping utility at 90.4%.

Methodology in Plain English

The benchmark is built in three stages.

Seed construction. The authors start from the VISPR dataset, which contains approximately 10K images with fine-grained privacy labels. An LLM-based filtering step selects images with high visual quality, accurate annotations, and rich social context, keeping one dominant sensitive label per image. This curation yields 2,318 informative images. For each image, an LLM brainstorms a pair of social scenarios — one where sharing the image is appropriate and one where it is not — each specified by standard CI parameters: data subject, sender, recipient, data type (image plus label), and transmission principle, plus a transmission method and contextual domain.

Story construction. Each seed is expanded into a six-sentence narrative that elaborates the CI parameters and includes one sentence explicitly describing a plausible task motivation for sharing the image. This is what creates the privacy–utility trade-off. An evaluator LLM scores each story on three principles against thresholds (θ₁ = 4 for P1, θ₂ = 4 for P2, θ₃ = 5 for P3); stories below threshold are critiqued, revised, and re-scored for up to three iterations, after which failing cases are discarded. Both members of a pair must pass or the whole image and its cases are dropped. This leaves 1,026 pairs of story-level cases. Refinement raises the share of cases above threshold from 20% to 56.8%.

Trace construction. Each refined story is turned into an executable tool-use trace by adapting the text-centric sandbox from PrivacyLens into a multimodal setting, with a custom Google Drive toolkit (search, read, upload, share) so the agent can retrieve and attach images. Each trace is truncated at the final data-transfer action.

Evaluation. Models first answer a binary Q&A probe about whether a given information flow is appropriate, scored with Accuracy, Precision, Recall, and F1. At the Trace tier, models also actually complete the task, and an LLM-as-a-judge determines whether the resulting action leaked sensitive information. Negative cases report Leakage Rate (aggregate, textual, visual) and a Help Score; positive cases report visual Utility Rate and a Help Score.

Quality control. Five annotators rated 50 randomly sampled final stories (25 positive, 25 negative) — 150 ratings total, three independent annotations per story — on a 5-point Likert scale along the three principles. Agreement (Gwet's AC1) was 0.733 for Semantic Utility, 0.680 for Contextual Appropriateness, and 0.103 for Story Realism. Human–AI match was 98%, 92%, and 84% respectively.

Setup. GPT-4o was used for benchmark construction and all automated evaluations via the Azure OpenAI API (version 2024-02-15-preview, temperature 0). Open-source models were served with vLLM on an NVIDIA RTX 6000 Pro GPU with 96 GB VRAM. Evaluated models: GPT-4o, GPT-5, Mistral Large 3, Gemma-3 (4B, 12B, 27B), InternVL3.5 (8B, 14B), and Qwen3-VL (4B, 8B, 30B-A3B).

Why This Matters

Impact on research. The paper argues that text-centric CI benchmarks give an incomplete picture of agent privacy behavior. Its paired design, which scores appropriate and inappropriate flows from the same image, makes it possible to measure over-refusal and oversharing as two sides of one trade-off rather than treating refusal as the only virtue. The reported probing–action gap also suggests that passing a privacy probe is not evidence that an agent will act safely.

Real-world applications:

• Personal assistants that handle screenshots, photos, and mixed-media documents on a user's behalf.
• Email and messaging agents that decide which attachments to send to which recipients.
• Enterprise agents operating under domain-specific norms in healthcare, finance, insurance, or education.
• Background-screening, hiring, and archival workflows where repurposing an image outside its original context can cause discrimination or norm violations.

Industry relevance. Tool-use agents with Drive-style file access are already deployed or prototyped. The finding that most leakage happens through direct attachment, and that a CI-based prompt outperforms refusal-based prompting on the privacy–utility balance, is directly actionable for teams shipping agent products.

Future Directions

• Cover cross-cultural and broader normative variation. The authors note MPCI-Bench spans 10 domains but does not fully capture cross-cultural differences in privacy expectations, nor all real-world domains and transmission principles.
• Move beyond prompt-based mitigation. The paper evaluates only inference-time interventions; it explicitly leaves training-time interventions, architectural changes, policy-aware tool gating, and reinforcement-learning-based alignment unexplored.
• Close the probing–action gap. The paper frames translating correct normative judgment into action-level control as the central unsolved problem.
• Test mitigation across more models and deployments. The mitigation study covers only Mistral-Large-3 and Qwen3-VL-8B, and the authors caution that effectiveness may vary across model families and deployment settings.

Target Audience

Privacy and AI-safety researchers studying contextual integrity or agent behavior; benchmark designers who need paired, multimodal, judgment-plus-action evaluation methodology; MLLM and agent developers who want concrete evidence about visual leakage and prompt-based mitigations; and policy, compliance, or trust-and-safety practitioners evaluating whether an agent's data-sharing behavior matches the norms of its deployment domain.

Authors’ abstract

As language-model agents evolve from passive chatbots into proactive assistants that handle personal data, evaluating their adherence to social norms becomes increasingly critical, often through the lens of Contextual Integrity (CI). However, existing CI benchmarks are largely text-centric and primarily emphasize negative refusal scenarios, overlooking multimodal privacy risks and the fundamental trade-off between privacy and utility. In this paper, we introduce MPCI-Bench, the first Multimodal Pairwise Contextual Integrity benchmark for evaluating privacy behavior in agentic settings. MPCI-Bench consists of paired positive and negative instances derived from the same visual source and instantiated across three tiers: normative Seed judgments, context-rich Story reasoning, and executable agent action Traces. Data quality is ensured through a Tri-Principle Iterative Refinement pipeline. Evaluations of state-of-the-art multimodal models reveal systematic failures to balance privacy and utility and a pronounced modality leakage gap, where sensitive visual information is leaked more frequently than textual information. We will open-source MPCI-Bench to facilitate future research on agentic CI.