mlsanityData to 7 Oct 2026

MultiAgent Collaboration Attack: Investigating Adversarial Attacks in Large Language Model Collaborations via Debate

arXiv 2406.14711 ↗First posted 20 Jun 2024cs.CL cs.AI cs.MAcited by 54

Not on the map. Why?

Cited by 39 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Large Language Models (LLMs) have shown exceptional results on current benchmarks when working individually. The advancement in their capabilities, along with a reduction in parameter size and inference times, has facilitated the use of these models as agents, enabling interactions among multiple models to execute complex tasks. Such collaborations offer several advantages, including the use of specialized models (e.g. coding), improved confidence through multiple computations, and enhanced divergent thinking, leading to more diverse outputs. Thus, the collaborative use of language models is expected to grow significantly in the coming years. In this work, we evaluate the behavior of a network of models collaborating through debate under the influence of an adversary. We introduce pertinent metrics to assess the adversary's effectiveness, focusing on system accuracy and model agreement. Our findings highlight the importance of a model's persuasive ability in influencing others. Additionally, we explore inference-time methods to generate more compelling arguments and evaluate the potential of prompt-based mitigation as a defensive strategy.

Key facts. Cited by 54 papers in the map (data through 7 Oct 2026). Builds on 16 of its 26 references that are in the map. First posted on arXiv 20 Jun 2024.

Builds on

16 of 26 references.

References · 16

  1. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  2. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  3. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  4. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
  5. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  6. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 995
  7. 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debatecited by 724
  8. 2023Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agentscited by 329
  9. 2023CodeGeeX: A Pre-Trained Model for Code Generation with Multilingual Benchmarking on HumanEval-Xcited by 294
  10. 2024Debating with More Persuasive LLMs Leads to More Truthful Answerscited by 255
  11. 2023Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology Viewcited by 221
  12. 2020Emergent Multi-Agent Communication in the Deep Learning Eracited by 168
4 more
  1. 2024On the Conversational Persuasiveness of Large Language Models: A Randomized Controlled Trialcited by 105
  2. 2024What Evidence Do Language Models Find Convincing?cited by 74
  3. 2024Shall We Team Up: Exploring Spontaneous Cooperation of Competing LLM Agentscited by 48
  4. 2024Can Language Models Recognize Convincing Arguments?cited by 21

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 21; 2026: 30.

54 papers.

Sort:

By area:

  1. 2024SWE-Search: Enhancing Software Agents with Monte Carlo Tree Search and Iterative Refinementcited by 124
  2. 2025Red-Teaming LLM Multi-Agent Systems via Communication Attackscited by 118
  3. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  4. 2025A Survey on Trustworthy LLM Agents: Threats and Countermeasurescited by 93
  5. 2024On the Resilience of LLM-Based Multi-Agent Collaboration with Faulty Agentscited by 92
  6. 2025Talk Isn't Always Cheap: Understanding Failure Modes in Multi-Agent Debatecited by 63
  7. 2025SAGA: A Security Architecture for Governing AI Agentic Systemscited by 45
  8. 2024Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)cited by 44
  9. 2025A Survey of Test-Time Compute: From Intuitive Inference to Deliberate Reasoningcited by 40
  10. 2026Memory Poisoning Attack and Defense on Memory Based LLM-Agentscited by 40
  11. 2025GUARDIAN: Safeguarding LLM Multi-Agent Collaborations with Temporal Graph Modelingcited by 35
  12. 2025textitAgents Under Siege: Breaking Pragmatic Multi-Agent LLM Systems with Optimized Prompt Attackscited by 31
38 more
  1. 2025Attack the Messages, Not the Agents: A Multi-round Adaptive Stealthy Tampering Framework for LLM-MAScited by 17
  2. 2025BlockA2A: Towards Secure and Verifiable Agent-to-Agent Interoperabilitycited by 14
  3. 2025Debate-to-Detect: Reformulating Misinformation Detection as a Real-World Debate with Large Language Modelscited by 12
  4. 2025An Adversary-Resistant Multi-Agent LLM System via Credibility Scoringcited by 10
  5. 2025Demonstrations of Integrity Attacks in Multi-Agent Systemscited by 7
  6. 2025SentinelNet: Safeguarding Multi-Agent Collaboration Through Credit-Based Dynamic Threat Detectioncited by 7
  7. 2026FlowSteer: Prompt-Only Workflow Steering Exposes Planning-Time Vulnerabilities in Multi-Agent LLM Systemscited by 7
  8. 2025MASTER: Multi-Agent Security Through Exploration of Roles and Topological Structures -- A Comprehensive Frameworkcited by 6
  9. 2025To trust or not to trust: Attention-based Trust Management for LLM Multi-Agent Systemscited by 6
  10. 2026Epistemic Context Learning: Building Trust the Right Way in LLM-Based Multi-Agent Systemscited by 6
  11. 2025PEAR: Planner-Executor Agent Robustness Benchmarkcited by 5
  12. 2026INFA-Guard: Mitigating Malicious Propagation via Infection-Aware Safeguarding in LLM-Based Multi-Agent Systemscited by 5
  13. 2026ACIArena: Toward Unified Evaluation for Agent Cascading Injectioncited by 5
  14. 2025Preventing Rogue Agents Improves Multi-Agent Collaborationcited by 3
  15. 2025MArgE: Meshing Argumentative Evidence from Multiple Large Language Models for Justifiable Claim Verificationcited by 3
  16. 2025Securing Multi-Agent Systems Against Corruptions via Node Contribution Backpropagationcited by 3
  17. 2025Don't Trust Your Upstream: Exploiting LLM Multi-Agent System via Topology-Guided Adversarial Propagationcited by 3
  18. 2026Architecture Matters for Multi-Agent Securitycited by 3
  19. 2026EquiMem: Calibrating Shared Memory in Multi-Agent Debate via Game-Theoretic Equilibriumcited by 3
  20. 2026Social Dynamics as Critical Vulnerabilities that Undermine Objective Decision-Making in LLM Collectivescited by 2
  21. 2026When Embedding-Based Defenses Fail: Rethinking Safety in LLM-Based Multi-Agent Systemscited by 2
  22. 2026MAGIQ: A Post-Quantum Multi-Agentic AI Governance System with Provable Securitycited by 2
  23. 2026Defending LLM-based Multi-Agent Systems Against Cooperative Attacks with Sentence-Level Rectificationcited by 2
  24. 2026Dive into Ambiguity: A*-Inspired Multi-Agents Commonsense Obfuscation Attack on LLM Promptscited by 2
  25. 2026Misinformation Propagation in Benign Multi-Agent Systemscited by 2
  26. 2026Social Networks of LLM Agentscited by 2
  27. 2026From Secure Agentic AI to Secure Agentic Web: Challenges, Threats, and Future Directionscited by 1
  28. 2026Insider Attacks in Multi-Agent LLM Consensus Systemscited by 1
  29. 2026GAMBIT: A Three-Mode Benchmark for Adversarial Robustness in Multi-Agent LLM Collectivescited by 1
  30. 2026Isolation as a First-Class Principle for LLM-Agent System Safety: Concepts, Taxonomy, Challenges and Future Directionscited by 1
  31. 2026Agent Harness Distillation: Inference-Time Harness Extraction and Exploitation in Autonomous Multi-Agent Systemscited by 1
  32. 2026Is Monitoring Enough? Strategic Agent Selection For Stealthy Attack in Multi-Agent Discussionscited by 0
  33. 2026You Only Align Once: Propagating Cooperative Behaviors in Multi-Agent Systems through Seed Agentscited by 0
  34. 2026Generative AI and Digital Ecosystem Resilience: A Proactive Lifecycle-Based Surveycited by 0
  35. 2026MAStrike: Shapley-Guided Collusive Red-Teaming on Multi-Agent Systemscited by 0
  36. 2026The Evaluator Is Part of the Experiment: Measuring Open-Ended LLM Conformitycited by 0
  37. 2026Connecting the Dots in Agentic AI Security: A Cross-Dimensional Threat Taxonomy, Evaluation Maturity, and Open Challengescited by 0
  38. 2026Concealing LLM-Based Multi-Agent Topology via Phantom Structure Injectioncited by 0

50 most cited of 54 All 54 (JSON)