mlsanityData to 7 Oct 2026

Red-Teaming LLM Multi-Agent Systems via Communication Attacks

arXiv 2502.14847 ↗First posted 20 Feb 2025cs.CRcited by 118

On the map: LLM agents and tool use → Agent threats, guardrails and governance

also in 1 more
Abstract (arXiv)

Large Language Model-based Multi-Agent Systems (LLM-MAS) have revolutionized complex problem-solving capability by enabling sophisticated agent collaboration through message-based communications. While the communication framework is crucial for agent coordination, it also introduces a critical yet unexplored security vulnerability. In this work, we introduce Agent-in-the-Middle (AiTM), a novel attack that exploits the fundamental communication mechanisms in LLM-MAS by intercepting and manipulating inter-agent messages. Unlike existing attacks that compromise individual agents, AiTM demonstrates how an adversary can compromise entire multi-agent systems by only manipulating the messages passing between agents. To enable the attack under the challenges of limited control and role-restricted communication format, we develop an LLM-powered adversarial agent with a reflection mechanism that generates contextually-aware malicious instructions. Our comprehensive evaluation across various frameworks, communication structures, and real-world applications demonstrates that LLM-MAS is vulnerable to communication-based attacks, highlighting the need for robust security measures in multi-agent systems.

Key facts. Cited by 118 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 26 of its 42 references that are in the map. First posted on arXiv 20 Feb 2025.

Builds on

26 of 42 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Breaking Agents: Compromising Autonomous LLM Agents Through Malfunction Amplification30 Jul
  2. 2024Flooding Spread of Manipulated Knowledge in LLM-Based Multi-Agent Communities10 Jul
  3. 2024PsySafe: A Comprehensive Framework for Psychological-based Attack, Defense, and Evaluation of Multi-agent System Safety22 Jan

Other references · 23

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2021Program Synthesis with Large Language Modelscited by 3,509
  6. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  7. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
  8. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  9. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
  10. 2023ChatDev: Communicative Agents for Software Developmentcited by 893
  11. 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debatecited by 724
  12. 2023Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agentscited by 329
11 more
  1. 2023MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoningcited by 311
  2. 2023S3: Social-network Simulation System with Large Language Model-Empowered Agentscited by 228
  3. 2023Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology Viewcited by 221
  4. 2024Large Language Model-Based Agents for Software Engineering: A Surveycited by 173
  5. 2023Language Agents with Reinforcement Learning for Strategic Play in the Werewolf Gamecited by 148
  6. 2024AgentScope: A Flexible yet Robust Multi-Agent Platformcited by 120
  7. 2023Avalon's Game of Thoughts: Battle Against Deception through Recursive Contemplationcited by 102
  8. 2023Examining Inter-Consistency of Large Language Models Collaboration: An In-depth Analysis via Debatecited by 94
  9. 2024MultiAgent Collaboration Attack: Investigating Adversarial Attacks in Large Language Model Collaborations via Debatecited by 54
  10. 2024Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)cited by 44
  11. 2024Make LLMs better zero-shot reasoners: Structure-orientated autonomous reasoningcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 47; 2026: 71.

118 papers.

Sort:

By area:

  1. 2025Large Language Model Agent: A Survey on Methodology, Applications and Challengescited by 194
  2. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  3. 2025A survey of agent interoperability protocols: Model Context Protocol (MCP), Agent Communication Protocol (ACP), Agent-to-Agent Protocol (A2A), and Agent Network Protocol (ANP)cited by 98
  4. 2025A Survey on Trustworthy LLM Agents: Threats and Countermeasurescited by 93
  5. 2025A Survey of LLM-Driven AI Agent Communication: Protocols, Security Risks, and Defense Countermeasurescited by 54
  6. 2025SAGA: A Security Architecture for Governing AI Agentic Systemscited by 45
  7. 2025Agentic Web: Weaving the Next Web with AI Agentscited by 43
  8. 2025GUARDIAN: Safeguarding LLM Multi-Agent Collaborations with Temporal Graph Modelingcited by 35
  9. 2025A Survey on Autonomy-Induced Security Risks in Large Model-Based Agentscited by 28
  10. 2025We Should Identify and Mitigate Third-Party Safety Risks in MCP-Powered Agent Systemscited by 27
  11. 2025Free-MAD: Consensus-Free Multi-Agent Debatecited by 27
  12. 2025Security Concerns for Large Language Models: A Surveycited by 26
38 more
  1. 2025Agentic AI Security: Threats, Defenses, Evaluation, and Open Challengescited by 26
  2. 2025TAMAS: Benchmarking Adversarial Risks in Multi-Agent LLM Systemscited by 24
  3. 2025Securing GenAI Multi-Agent Systems Against Tool Squatting: A Zero Trust Registry-Based Approachcited by 17
  4. 2025Attack the Messages, Not the Agents: A Multi-round Adaptive Stealthy Tampering Framework for LLM-MAScited by 17
  5. 2025A Survey on Agentic Security: Applications, Threats and Defensescited by 17
  6. 2025SafeScientist: Toward Risk-Aware Scientific Discoveries by LLM Agentscited by 14
  7. 2026From Spark to Fire: Modeling and Mitigating Error Cascades in LLM-Based Multi-Agent Collaborationcited by 13
  8. 2026ARGUS: Defending LLM Agents Against Context-Aware Prompt Injectioncited by 13
  9. 2025Inherent and emergent liability issues in LLM-based agentic systems: a principal-agent perspectivecited by 12
  10. 2025Who's the Mole? Modeling and Detecting Intention-Hiding Malicious Agents in LLM-Based Multi-Agent Systemscited by 12
  11. 2025MAD-Spear: A Conformity-Driven Prompt Injection Attack on Multi-Agent Debate Systemscited by 10
  12. 2025Terrarium: Revisiting the Blackboard for Multi-Agent Safety, Privacy, and Security Studiescited by 10
  13. 2025MedSentry: Understanding and Mitigating Safety Risks in Medical LLM Multi-Agent Systemscited by 8
  14. 2025Web Fraud Attacks Against LLM-Driven Multi-Agent Systemscited by 8
  15. 2025Distributional AGI Safetycited by 8
  16. 2025Goal-Aware Identification and Rectification of Misinformation in Multi-Agent Systemscited by 7
  17. 2025Demonstrations of Integrity Attacks in Multi-Agent Systemscited by 7
  18. 2025SentinelNet: Safeguarding Multi-Agent Collaboration Through Credit-Based Dynamic Threat Detectioncited by 7
  19. 2026Thought Virus: Viral Misalignment via Subliminal Prompting in Multi-Agent Systemscited by 7
  20. 2026FlowSteer: Prompt-Only Workflow Steering Exposes Planning-Time Vulnerabilities in Multi-Agent LLM Systemscited by 7
  21. 2025To trust or not to trust: Attention-based Trust Management for LLM Multi-Agent Systemscited by 6
  22. 2025Agentic Services Computingcited by 6
  23. 2025Security Analysis of Agentic AI Communication Protocols: A Comparative Evaluationcited by 6
  24. 2025PEAR: Planner-Executor Agent Robustness Benchmarkcited by 5
  25. 2025Formalizing the Safety, Security, and Functional Properties of Agentic AI Systemscited by 5
  26. 2026INFA-Guard: Mitigating Malicious Propagation via Infection-Aware Safeguarding in LLM-Based Multi-Agent Systemscited by 5
  27. 2026ACIArena: Toward Unified Evaluation for Agent Cascading Injectioncited by 5
  28. 2026PAC-BENCH: Evaluating Multi-Agent Collaboration under Privacy Constraintscited by 5
  29. 2026Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluationcited by 5
  30. 2025Evo-MARL: Co-Evolutionary Multi-Agent Reinforcement Learning for Internalized Safetycited by 4
  31. 2025Scheming Ability in LLM-to-LLM Strategic Interactionscited by 4
  32. 2026GroupGuard: A Framework for Modeling and Defending Collusive Attacks in Multi-Agent Systemscited by 4
  33. 2025A Survey on AgentOps: Categorization, Challenges, and Future Directionscited by 3
  34. 2025AdvEvo-MARL: Shaping Internalized Safety through Adversarial Co-Evolution in Multi-Agent Reinforcement Learningcited by 3
  35. 2025Securing Multi-Agent Systems Against Corruptions via Node Contribution Backpropagationcited by 3
  36. 2025The Subtle Art of Defection: Understanding Uncooperative Behaviors in LLM based Multi-Agent Systemscited by 3
  37. 2025MAS-Shield: A Defense Framework for Secure and Efficient LLM MAScited by 3
  38. 2025Don't Trust Your Upstream: Exploiting LLM Multi-Agent System via Topology-Guided Adversarial Propagationcited by 3

50 most cited of 118 All 118 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026CIA: Inferring the Communication Topology from LLM-based Multi-Agent Systems
  2. 2025textitAgents Under Siege: Breaking Pragmatic Multi-Agent LLM Systems with Optimized Prompt Attacks
  3. 2025Beyond Self-Talk: A Communication-Centric Survey of LLM-Based Multi-Agent Systems
  4. 2025The Dark Side of LLMs: Agent-based Attack Vectors for System-level Compromise
  5. 2026Insider Attacks in Multi-Agent LLM Consensus Systems
  6. 2024Prompt Infection: LLM-to-LLM Prompt Injection within Multi-Agent Systems
  7. 2025A Multi-Agent LLM Defense Pipeline Against Prompt Injection Attacks
  8. 2025Comprehensive Vulnerability Analysis is Necessary for Trustworthy LLM-MAS