# MAGPIE: A benchmark for Multi-AGent contextual PrIvacy Evaluation

arXiv: https://arxiv.org/abs/2510.15186 · HTML: https://mlsanity.com/paper/2510.15186/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 30 of its 48 references that are in the map.
- First posted on arXiv 16 Oct 2025.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)

## Builds on (most cited first, 30 of 30)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation](https://mlsanity.com/paper/2308.08155/) (2023)
- [Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection](https://mlsanity.com/paper/2302.12173/) (2023)
- [Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory](https://mlsanity.com/paper/2310.17884/) (2023)
- [Unveiling Privacy Risks in LLM Agent Memory](https://mlsanity.com/paper/2502.13172/) (2025)
- [LLMs know their vulnerabilities: Uncover Safety Gaps through Natural Distribution Shifts](https://mlsanity.com/paper/2410.10700/) (2024)
- [IPIGuard: A Novel Tool Dependency Graph-Based Defense Against Indirect Prompt Injection in LLM Agents](https://mlsanity.com/paper/2508.15310/) (2025)
- [SiriuS: Self-improving Multi-agent Systems via Bootstrapped Reasoning](https://mlsanity.com/paper/2502.04780/) (2025)
- [PC-Agent: A Hierarchical Multi-Agent Collaboration Framework for Complex Task Automation on PC](https://mlsanity.com/paper/2502.14282/) (2025)
- [INMS: Memory Sharing for Large Language Model based Agents](https://mlsanity.com/paper/2404.09982/) (2024)
- [LLM Agents Should Employ Security Principles](https://mlsanity.com/paper/2505.24019/) (2025)
- [Simple Prompt Injection Attacks Can Leak Personal Data Observed by LLM Agents During Task Execution](https://mlsanity.com/paper/2506.01055/) (2025)
- [SAFEFLOW: A Principled Protocol for Trustworthy and Transactional Autonomous Agent Systems](https://mlsanity.com/paper/2506.07564/) (2025)
- [Searching for Privacy Risks in LLM Agents via Simulation](https://mlsanity.com/paper/2508.10880/) (2025)
- [SoK: The Privacy Paradox of Large Language Models: Advancements, Privacy Risks, and Mitigation](https://mlsanity.com/paper/2506.12699/) (2025)
- [1-2-3 Check: Enhancing Contextual Privacy in LLM via Multi-Agent Reasoning](https://mlsanity.com/paper/2508.07667/) (2025)
- [To Protect the LLM Agent Against the Prompt Injection Attack with Polymorphic Prompt](https://mlsanity.com/paper/2506.05739/) (2025)
- [LLM-Based Multi-Agent Blackboard System for Information Discovery in Data Science](https://mlsanity.com/paper/2510.01285/) (2025)
- [Multi-Agent Systems Powered by Large Language Models: Applications in Swarm Intelligence](https://mlsanity.com/paper/2503.03800/) (2025)
- [Efficient Multi-Agent Collaboration with Tool Use for Online Planning in Complex Table Question Answering](https://mlsanity.com/paper/2412.20145/) (2024)
- [Prεεmpt: Sanitizing Sensitive Prompts for LLMs](https://mlsanity.com/paper/2504.05147/) (2025)
- [Exploring Advanced LLM Multi-Agent Systems Based on Blackboard Architecture](https://mlsanity.com/paper/2507.01701/) (2025)
- [MAEBE: Multi-Agent Emergent Behavior Framework](https://mlsanity.com/paper/2506.03053/) (2025)
- [Architecting Resilient LLM Agents: A Guide to Secure Plan-then-Execute Implementations](https://mlsanity.com/paper/2509.08646/) (2025)
- [Language-Driven Coordination and Learning in Multi-Agent Simulation Environments](https://mlsanity.com/paper/2506.04251/) (2025)
- [Enterprise AI Must Enforce Participant-Aware Access Control](https://mlsanity.com/paper/2509.14608/) (2025)
- [The Social Laboratory: A Psychometric Framework for Multi-Agent LLM Evaluation](https://mlsanity.com/paper/2510.01295/) (2025)

## Built on by (most cited first, 29 of 29)

- [Agent Tools Orchestration Leaks More: Dataset, Benchmark, and Mitigation](https://mlsanity.com/paper/2512.16310/) (2025)
- [MAESTRO: Multi-Agent Evaluation Suite for Testing, Reliability, and Observability](https://mlsanity.com/paper/2601.00481/) (2026)
- [AgentSocialBench: Evaluating Privacy Risks in Human-Centered Agentic Social Networks](https://mlsanity.com/paper/2604.01487/) (2026)
- [LCGuard: Latent Communication Guard for Safe KV Sharing in Multi-Agent Systems](https://mlsanity.com/paper/2605.22786/) (2026)
- [Taxonomy and Consistency Analysis of Safety Benchmarks for AI Agents](https://mlsanity.com/paper/2605.16282/) (2026)
- [MuPPET: A Benchmark for Contextual Privacy of LLM Assistants in Multi-Party Conversations](https://mlsanity.com/paper/2606.23217/) (2026)
- [Whose Side Is Your Agent On? Multi-Party Principal Loyalty in LLM Agents](https://mlsanity.com/paper/2606.30383/) (2026)
- [PrivacyAlign: Contextual Privacy Alignment for LLM Agents](https://mlsanity.com/paper/2606.21710/) (2026)
- [PiSAs: Benchmarking Contextual Integrity in Multi-User Agentic Systems](https://mlsanity.com/paper/2607.05318/) (2026)
- [Beyond Goodhart's Law: A Dynamic Benchmark for Evaluating Compliance in Multi-Agent Systems](https://mlsanity.com/paper/2606.07805/) (2026)
- [The Interlocutor Effect: Why LLMs Leak More Personal Data to Agents Than Humans](https://mlsanity.com/paper/2606.09844/) (2026)
- [TRAP: Benchmark for Task-completion and Resistance to Active Privacy-extraction](https://mlsanity.com/paper/2606.18996/) (2026)
- [SovereignNegotiation-Bench: Evaluating User-Owned Personal Agents In Delegated Bargaining Under Privacy, Consent, Evidence, And Institutional Pressure](https://mlsanity.com/paper/2607.02814/) (2026)
- [Behavioral Privacy Leakage in Agentic Negotiation: Formalizing and Mitigating Inference Attacks via Randomized Policies](https://mlsanity.com/paper/2607.06815/) (2026)
- [WeClawArena: An Auditable Sandbox and Benchmark for Cross-User Agents Collaboration and Security in Human-Centered Agent Networks](https://mlsanity.com/paper/2608.03499/) (2026)
- [Authorization Before Context: A Model-Neutral Audience Boundary Against Cross-Audience Memory Leakage in Agentic Systems](https://mlsanity.com/paper/2608.17148/) (2026)
- [SoK: When Safe Agents Fail Together: The Security of Multi Agent LLM Systems](https://mlsanity.com/paper/2609.00595/) (2026)
- [Agents That Know Too Much: A Data-Centric Survey of Privacy in LLM Agents](https://mlsanity.com/paper/2606.26627/) (2026)
- [Can LLM Agents Price Competitively? A Dynamic Multi-Attribute Auction Benchmark for Agentic Commerce](https://mlsanity.com/paper/2608.00102/) (2026)
- [From Monoliths to Swarms: A Study of Attack Surface Evolution in the Transition to Multi-Agent Web Systems](https://mlsanity.com/paper/2608.00202/) (2026)
- [MNC: Scope-Bound Semantic Declassification for Private LLM-Agent Communication](https://mlsanity.com/paper/2608.01719/) (2026)
- [MemArena: An Ego-Centric Benchmark for On-Device Agentic Personal Memory Assistants at Scale](https://mlsanity.com/paper/2608.02613/) (2026)
- [From Passive Delegates to Strategic Negotiators: Reinforcing Social Reasoning in Small Language Models with SocialRL](https://mlsanity.com/paper/2608.13787/) (2026)
- [PriMobiBench: Characterizing Visual Privacy Leakage in VLM-Driven Mobile GUI Agents](https://mlsanity.com/paper/2609.13873/) (2026)
- [Confuse the Model, Control the Flow: Understanding and Mitigating Privacy Leakage from LLM Agents with Information Flow Control](https://mlsanity.com/paper/2609.14003/) (2026)
- [CoSec: Benchmarking Agent Security in Communities](https://mlsanity.com/paper/2609.34790/) (2026)
- [EP-Mem: Elastic Privacy Memory for Social Relationship-Aware LLM Agents](https://mlsanity.com/paper/2609.35233/) (2026)
- [HarnessNet: Feedback-driven Hierarchical Joint Harness Evolution for Multi-agent Systems](https://mlsanity.com/paper/2610.15221/) (2026)
- [When Agents Talk, Secrets Travel: Topology-Driven Leakage and Recipient-Conditioned Sanitization in Multi-Agent LLM Systems](https://mlsanity.com/paper/2610.16039/) (2026)

## Abstract (arXiv)

A core challenge for autonomous LLM agents in collaborative settings is balancing robust privacy understanding and preservation alongside task efficacy. Existing privacy benchmarks only focus on simplistic, single-turn interactions where private information can be trivially omitted without affecting task outcomes. In this paper, we introduce MAGPIE (Multi-AGent contextual PrIvacy Evaluation), a novel benchmark of 200 high-stakes tasks designed to evaluate privacy understanding and preservation in multi-agent collaborative, non-adversarial scenarios. MAGPIE integrates private information as essential for task resolution, forcing agents to balance effective collaboration with strategic information control. Our evaluation reveals that state-of-the-art agents, including GPT-5 and Gemini 2.5-Pro, exhibit significant privacy leakage, with Gemini 2.5-Pro leaking up to 50.7% and GPT-5 up to 35.1% of the sensitive information even when explicitly instructed not to. Moreover, these agents struggle to achieve consensus or task completion and often resort to undesirable behaviors such as manipulation and power-seeking (e.g., Gemini 2.5-Pro demonstrating manipulation in 38.2% of the cases). These findings underscore that current LLM agents lack robust privacy understanding and are not yet adequately aligned to simultaneously preserve privacy and maintain effective collaboration in complex environments.