IP Leakage Attacks Targeting LLM-Based Multi-Agent Systems
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
The rapid advancement of Large Language Models (LLMs) has led to the emergence of Multi-Agent Systems (MAS) to perform complex tasks through collaboration. However, the intricate nature of MAS, including their architecture and agent interactions, raises significant concerns regarding intellectual property (IP) protection. In this paper, we introduce MASLEAK, a novel attack framework designed to extract sensitive information from MAS applications. MASLEAK targets a practical, black-box setting, where the adversary has no prior knowledge of the MAS architecture or agent configurations. The adversary can only interact with the MAS through its public API, submitting attack query q and observing outputs from the final agent. Inspired by how computer worms propagate and infect vulnerable network hosts, MASLEAK carefully crafts adversarial query q to elicit, propagate, and retain responses from each MAS agent that reveal a full set of proprietary components, including the number of agents, system topology, system prompts, task instructions, and tool usages. We construct the first synthetic dataset of MAS applications with 810 applications and also evaluate MASLEAK against real-world MAS applications, including Coze and CrewAI. MASLEAK achieves high accuracy in extracting MAS IP, with an average attack success rate of 87% for system prompts and task instructions, and 92% for system architecture in most cases. We conclude by discussing the implications of our findings and the potential defenses.
Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 60 of its 81 references that are in the map. First posted on arXiv 18 May 2025.
Builds on
60 of 81 references.
Closest ancestors · 9
Same direction, newest first
- 2025textitAgents Under Siege: Breaking Pragmatic Multi-Agent LLM Systems with Optimized Prompt Attacks31 Mar
- 2024LeakAgent: RL-based Red-teaming Agent for LLM Privacy Leakage7 Dec
- 2024NetSafe: Exploring the Topological Safety of Multi-agent Networks21 Oct
- 2024AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents11 Oct
- 2024Prompt Infection: LLM-to-LLM Prompt Injection within Multi-Agent Systems9 Oct
- 2024Breaking Agents: Compromising Autonomous LLM Agents Through Malfunction Amplification30 Jul
- 2024RedAgent: Red Teaming Large Language Models with Context-aware Autonomous Language Agent23 Jul
- 2024Flooding Spread of Manipulated Knowledge in LLM-Based Multi-Agent Communities10 Jul
- 2024PsySafe: A Comprehensive Framework for Psychological-based Attack, Defense, and Evaluation of Multi-agent System Safety22 Jan
Other references · 51
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
- 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
- 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
- 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,808
- 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,356
- 2023ChatDev: Communicative Agents for Software Developmentcited by 919
- 2016Practical Black-Box Attacks against Machine Learningcited by 903
- 2021Membership Inference Attacks From First Principlescited by 878
- 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 832
38 more
- 2024Large Language Model based Multi-Agents: A Survey of Progress and Challengescited by 769
- 2016Stealing Machine Learning Models via Prediction APIscited by 619
- 2025Why Do Multi-Agent LLM Systems Fail?cited by 604
- 2021FinQA: A Dataset of Numerical Reasoning over Financial Datacited by 562
- 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 434
- 2024StruQ: Defending Against Prompt Injection with Structured Queriescited by 375
- 2023Formalizing and Benchmarking Prompt Injection Attacks and Defensescited by 364
- 2018Knockoff Nets: Stealing Functionality of Black-Box Modelscited by 292
- 2021Enhanced Membership Inference Attacks against Machine Learning Modelscited by 248
- 2024Scaling Large Language Model-based Multi-Agent Collaborationcited by 241
- 2024Tool Learning with Large Language Models: A Surveycited by 223
- 2023AutoAgents: A Framework for Automatic Agent Generationcited by 216
- 2019High Accuracy and High Fidelity Extraction of Neural Networkscited by 214
- 2024Stealing Part of a Production Language Modelcited by 159
- 2024Optimization-based Prompt Injection Attack to LLM-as-a-Judgecited by 143
- 2024InFoBench: Evaluating Instruction Following Ability in Large Language Modelscited by 135
- 2024PLeak: Prompt Leaking Attacks against Large Language Model Applicationscited by 127
- 2022Membership Inference Attacks by Exploiting Loss Trajectorycited by 101
- 2024On the Resilience of LLM-Based Multi-Agent Collaboration with Faulty Agentscited by 99
- 2023Effective Prompt Extraction from Language Modelscited by 85
- 2023Split and Merge: Aligning Position Biases in LLM-based Evaluatorscited by 72
- 2024SelfDefend: LLMs Can Defend Themselves against Jailbreaking in a Practical Mannercited by 63
- 2024Here Comes The AI Worm: Unleashing Zero-click Worms that Target GenAI-Powered Applicationscited by 59
- 2025Stop Overvaluing Multi-Agent Debate -- We Must Rethink Evaluation and Embrace Model Heterogeneitycited by 44
- 2024Membership Inference Attacks Against In-Context Learningcited by 43
- 2024Extracting Prompts by Inverting LLM Outputscited by 41
- 2024LLM-PBE: Assessing Data Privacy in Large Language Modelscited by 41
- 2024Prompt Stealing Attacks Against Large Language Modelscited by 40
- 2022CCTEST: Testing and Repairing Code Completion Systemscited by 35
- 2024ChatBug: A Common Vulnerability of Aligned LLMs Induced by Chat Templatescited by 30
- 2024ToolBeHonest: A Multi-level Hallucination Diagnostic Benchmark for Tool-Augmented Large Language Modelscited by 25
- 2023Refining Decompiled C Code with Large Language Modelscited by 24
- 2024API-guided Dataset Synthesis to Finetune Large Code Modelscited by 11
- 2022Unleashing the Power of Compiler Intermediate Representation to Enhance Neural Program Embeddingscited by 10
- 2025Low-Cost and Comprehensive Non-textual Input Fuzzing with LLM-Synthesized Input Generatorscited by 10
- 2023REEF: A Framework for Collecting Real-World Vulnerabilities and Fixescited by 8
- 2025When Grammar Guides the Attack: Uncovering Control-Plane Vulnerabilities in LLMs with Structured Outputcited by 4
- 2024SPVR: syntax-to-prompt vulnerability repair based on large language modelscited by 3
50 most cited of 51 All 51 (JSON)
Built on by
Citing papers per year: 2025: 2; 2026: 6.
8 papers.
- 2025A Survey of LLM-Driven AI Agent Communication: Protocols, Security Risks, and Defense Countermeasurescited by 57
- 2026On Protecting Agentic Systems' Intellectual Property via Watermarkingcited by 8
- 2026FlowSteer: Prompt-Only Workflow Steering Exposes Planning-Time Vulnerabilities in Multi-Agent LLM Systemscited by 7
- 2025Don't Trust Your Upstream: Exploiting LLM Multi-Agent System via Topology-Guided Adversarial Propagationcited by 5
- 2026Sequential Behavioral Watermarking for LLM Agentscited by 5
- 2026Clawed and Dangerous: Can We Trust Open Agentic Systems?cited by 3
- 2026When Prompts Control Robots: Prompt Injection Attacks in Multi-Agent Robotic Systemscited by 1
- 2026ToolMinimize: Auditing and Rewriting LLM Agent Tool Calls to Minimize Privacy Exposurecited by 1
- 2026ToolMinimize: Auditing and Rewriting LLM Agent Tool Calls to Minimize Privacy Exposure25 Aug 2026
- 2026When Prompts Control Robots: Prompt Injection Attacks in Multi-Agent Robotic Systems1 Aug 2026
- 2026FlowSteer: Prompt-Only Workflow Steering Exposes Planning-Time Vulnerabilities in Multi-Agent LLM Systems12 May 2026
- 2026Sequential Behavioral Watermarking for LLM Agents11 May 2026
- 2026Clawed and Dangerous: Can We Trust Open Agentic Systems?27 Mar 2026
- 2026On Protecting Agentic Systems' Intellectual Property via Watermarking9 Feb 2026
- 2025Don't Trust Your Upstream: Exploiting LLM Multi-Agent System via Topology-Guided Adversarial Propagation3 Dec 2025
- 2025A Survey of LLM-Driven AI Agent Communication: Protocols, Security Risks, and Defense Countermeasures24 Jun 2025
Similar, not cited
Close in wording, no citation either way
- 2025Securing Multi-Agent Systems Against Corruptions via Node Contribution Backpropagation
- 2025From Prompt Injections to Protocol Exploits: Threats in LLM-Powered AI Agents Workflows
- 2025The Dark Side of LLMs: Agent-based Attack Vectors for System-level Compromise
- 2025Multi-Agent Systems Execute Arbitrary Malicious Code
- 2025Comprehensive Vulnerability Analysis is Necessary for Trustworthy LLM-MAS
- 2025Automating Prompt Leakage Attacks on Large Language Models Using Agentic Approach
- 2025Attack the Messages, Not the Agents: A Multi-round Adaptive Stealthy Tampering Framework for LLM-MAS
- 2025Commercial LLM Agents Are Already Vulnerable to Simple Yet Dangerous Attacks