mlsanityData to 7 Oct 2026

AgentLeak: A Benchmark for Internal-Channel Privacy Leakage in Multi-Agent LLM Systems

arXiv 2602.11510 ↗First posted 12 Feb 2026cs.AIcited by 20

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

Multi-agent Large Language Model (LLM) systems create privacy risks that current output-only benchmarks cannot measure. When agents coordinate on tasks, sensitive data may pass through inter-agent messages, shared memory, and tool arguments, all pathways that final-output audits typically do not inspect. We introduce AgentLeak, a benchmark for evaluating internal-channel privacy leakage in multi-agent LLM systems. AgentLeak instruments seven privacy-relevant communication pathways and provides a large-scale empirical evaluation focused on final outputs, inter-agent messages, and shared memory. Across 1,000 scenarios spanning healthcare, finance, legal, and corporate domains, five production LLMs (GPT-4o, GPT-4o-mini, Claude 3.5 Sonnet, Mistral Large, and Llama 3.3 70B), and 4,979 validated execution traces, we find that multi-agent configurations reduce final-output leakage (C1: 27.2% vs 43.2% in single-agent mode) compared with single-agent baselines but introduce internal channels that raise total system exposure to 68.9% (aggregated across C1, C2, C5). Inter-agent messages (C2) leak at 68.8%, compared with 27.2% for final outputs (C1), meaning that output-only audits miss 41.7% of violations. Across all five models and four domains, the pattern C2 ≥ C1 holds consistently. These results suggest, within the evaluated coordinator-worker setting, that privacy risk in multi-agent systems is strongly shaped by architectural coordination channels rather than final-output behavior alone: it arises from internal channels that remain invisible to standard output-level defenses.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 27 of its 42 references that are in the map. First posted on arXiv 12 Feb 2026.

Builds on

27 of 42 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Agent Tools Orchestration Leaks More: Dataset, Benchmark, and Mitigation18 Dec
  2. 2025AgentDAM: Privacy Leakage Evaluation for Autonomous Web Agents12 Mar
  3. 2024Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents3 Oct
  4. 2024AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases17 Jul
  5. 2024AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents19 Jun
  6. 2024AirGapAgent: Protecting Privacy-Conscious Conversational Agents8 May

Other references · 21

  1. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  2. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  3. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  4. 2020Extracting Training Data from Large Language Modelscited by 2,144
  5. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  6. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
  7. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  8. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  9. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  10. 2022Red Teaming Language Models with Language Modelscited by 983
  11. 2023Prompt Injection attack against LLM-integrated Applicationscited by 568
  12. 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 505
9 more
  1. 2023NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Railscited by 347
  2. 2024TrustLLM: Trustworthiness in Large Language Modelscited by 280
  3. 2023Analyzing Leakage of Personally Identifiable Information in Language Modelscited by 278
  4. 2024Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Securitycited by 257
  5. 2024PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Actioncited by 161
  6. 2023Ignore This Title and HackAPrompt: Exposing Systemic Vulnerabilities of LLMs through a Global Scale Prompt Hacking Competitioncited by 65
  7. 2025Leaky Thoughts: Large Reasoning Models Are Not Private Thinkerscited by 25
  8. 2020Operationalizing the Legal Principle of Data Minimization for Personalizationcited by 19
  9. 2024MPC-Minimized Secure LLM Inferencecited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 20.

20 papers.

Sort:

By area:

  1. 2026A Survey on Long-Term Memory Security in LLM Agents: Attacks, Defenses, and Governance Across the Memory Lifecyclecited by 21
  2. 2026Auditing Agent Harness Safetycited by 8
  3. 2026Safety in Self-Evolving LLM Agent Systems: Threats, Amplification, and Case Studiescited by 6
  4. 2026ToolPrivacyBench: Benchmarking Purpose-Bound Privacy in Tool-Using LLM Agentscited by 6
  5. 2026MasDrift: Benchmarking Authorization Preservation Across Multi-Agent Architecturescited by 3
  6. 2026Capable but Careless: Do Computer-Use Agents Follow Contextual Integrity?cited by 2
  7. 2026When Latent Agents Lie: KV-Cache Integrity in Multi-Agent LLM Collaborationcited by 2
  8. 2026SovereignNegotiation-Bench: Evaluating User-Owned Personal Agents In Delegated Bargaining Under Privacy, Consent, Evidence, And Institutional Pressurecited by 1
  9. 2026PiSAs: Benchmarking Contextual Integrity in Multi-User Agentic Systemscited by 1
  10. 2026SoK: When Safe Agents Fail Together: The Security of Multi Agent LLM Systemscited by 1
  11. 2026Covert Assistance: Helpful LLM Agents Evade Oversight in Multi-Agent Systemscited by 1
  12. 2026StealthBench: Measuring Operational Stealth in Autonomous Offensive-Security Agentscited by 0
8 more
  1. 2026From Single Chatbots to Governed Agent Ecosystems: An Agentic AI Pattern Catalogue and Orchestration Framework for Mission-Critical Hospital Information Management Systemscited by 0
  2. 2026Facts Without Rules: Boundary Metadata Collapse in Multi-Agent LLM Handoffscited by 0
  3. 2026MOLE: Detecting Insider Threats in AI Agentscited by 0
  4. 2026ASLEval: Measuring Privacy Exposure Displacement in LLM Agent Sessionscited by 0
  5. 2026PII-TRACE: A Benchmark for Context-Aware PII Detection in Multi-Turn LLM Conversationscited by 0
  6. 2026PrivacySkills: How Privacy Guidance Shapes Source Selection in LLM Agentscited by 0
  7. 2026Privacy in Personalized AI Is a System Property, Not Just a Model Propertycited by 0
  8. 2026Deny Without Disabling: Authorization-Paired Evaluation and Control for Multi-Agent Systemscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Observable Channels, Not Just Storage: Evaluating Privacy Leakage in LLM Agent Pipelines
  2. 2026CIPL: A Channel-Aware Framework for Recoverable Privacy Leakage in LLM Agents
  3. 2026Information-Theoretic Privacy Control for Sequential Multi-Agent LLM Systems
  4. 20251-2-3 Check: Enhancing Contextual Privacy in LLM via Multi-Agent Reasoning
  5. 2026Got a Secret? LLM Agents Can't Keep It: Evaluating Privacy in Multi-Agent Systems
  6. 2025Privacy in Action: Towards Realistic Privacy Mitigation and Evaluation for LLM-Powered Agents
  7. 2024Whispers in the Machine: Confidentiality in Agentic Systems
  8. 2026OCELOT: Inference-Leakage Budgets for Privacy-Preserving LLM Agents