mlsanityData to 14 Oct 2026

Amplified Vulnerabilities: Structured Jailbreak Attacks on LLM-based Multi-Agent Debate

arXiv 2504.16489 ↗First posted 23 Apr 2025cs.CR cs.AIcited by 10

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 2 more
Abstract (arXiv)

Multi-Agent Debate (MAD), leveraging collaborative interactions among Large Language Models (LLMs), aim to enhance reasoning capabilities in complex tasks. However, the security implications of their iterative dialogues and role-playing characteristics, particularly susceptibility to jailbreak attacks eliciting harmful content, remain critically underexplored. This paper systematically investigates the jailbreak vulnerabilities of four prominent MAD frameworks built upon leading commercial LLMs (GPT-4o, GPT-4, GPT-3.5-turbo, and DeepSeek) without compromising internal agents. We introduce a novel structured prompt-rewriting framework specifically designed to exploit MAD dynamics via narrative encapsulation, role-driven escalation, iterative refinement, and rhetorical obfuscation. Our extensive experiments demonstrate that MAD systems are inherently more vulnerable than single-agent setups. Crucially, our proposed attack methodology significantly amplifies this fragility, increasing average harmfulness from 28.14% to 80.34% and achieving attack success rates as high as 80% in certain scenarios. These findings reveal intrinsic vulnerabilities in MAD architectures and underscore the urgent need for robust, specialized defenses prior to real-world deployment.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 45 of its 48 references that are in the map. First posted on arXiv 23 Apr 2025.

Builds on

45 of 48 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Stop Overvaluing Multi-Agent Debate -- We Must Rethink Evaluation and Embrace Model Heterogeneity12 Feb
  2. 2025S2-MAD: Breaking the Token Barrier to Enhance Multi-Agent Debate Efficiency7 Feb
  3. 2024GroupDebate: Enhancing the Efficiency of Multi-Agent Debate Using Group Discussion21 Sep
  4. 2024Improving Multi-Agent Debate with Sparse Communication Topology17 Jun
  5. 2023Learning to Break: Knowledge-Enhanced Reasoning in Multi-Agent Debate System8 Dec
  6. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate30 May

Other references · 39

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  5. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
  6. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  7. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
  8. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,154
  9. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 785
  10. 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debatecited by 742
  11. 2023AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviorscited by 598
  12. 2023"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Modelscited by 489
27 more
  1. 2023Detecting Language Model Attacks with Perplexitycited by 431
  2. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Ciphercited by 423
  3. 2023Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructionscited by 387
  4. 2023Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attackscited by 278
  5. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Surveycited by 230
  6. 2023Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulationcited by 214
  7. 2023Automatically Auditing Large Language Models via Discrete Optimizationcited by 191
  8. 2024Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fastcited by 153
  9. 2024Prompt Infection: LLM-to-LLM Prompt Injection within Multi-Agent Systemscited by 133
  10. 2024The Emerged Security and Privacy of LLM Agent: A Survey with Case Studiescited by 131
  11. 2024On Prompt-Driven Safeguarding for Large Language Modelscited by 116
  12. 2024How Alignment and Jailbreak Work: Explain LLM Safety through Intermediate Hidden Statescited by 98
  13. 2024GradSafe: Detecting Jailbreak Prompts for LLMs via Safety-Critical Gradient Analysiscited by 96
  14. 2024Internet of Agents: Weaving a Web of Heterogeneous Agents for Collaborative Intelligencecited by 86
  15. 2024Flooding Spread of Manipulated Knowledge in LLM-Based Multi-Agent Communitiescited by 86
  16. 2023Exchange-of-Thought: Enhancing Large Language Model Capabilities through Cross-Model Communicationcited by 82
  17. 2024Breaking Agents: Compromising Autonomous LLM Agents Through Malfunction Amplificationcited by 81
  18. 2024PsySafe: A Comprehensive Framework for Psychological-based Attack, Defense, and Evaluation of Multi-agent System Safetycited by 80
  19. 2023Attack Prompt Generation for Red Teaming and Defending Large Language Modelscited by 76
  20. 2024CodeChameleon: Personalized Encryption Framework for Jailbreaking Large Language Modelscited by 76
  21. 2024Play Guessing Game with LLM: Indirect Jailbreak Attack with Implicit Cluescited by 68
  22. 2024PRP: Propagating Universal Perturbations to Attack Large Language Model Guard-Railscited by 52
  23. 2025textitAgents Under Siege: Breaking Pragmatic Multi-Agent LLM Systems with Optimized Prompt Attackscited by 34
  24. 2024JailbreakEval: An Integrated Toolkit for Evaluating Jailbreak Attempts Against Large Language Modelscited by 31
  25. 2024Is poisoning a real threat to LLM alignment? Maybe more so than you thinkcited by 28
  26. 2024Foot In The Door: Understanding Large Language Model Jailbreaking via Cognitive Psychologycited by 27
  27. 2025Stay Focused: Problem Drift in Multi-Agent Debatecited by 23

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 6.

10 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025Echoes of Human Malice in Agents: Benchmarking LLMs for Multi-Turn Online Harassment Attacks
  2. 2024MRJ-Agent: An Effective Jailbreak Agent for Multi-Round Dialogue
  3. 2025Evaluating Multi-Agent Defences Against Jailbreaking Attacks on Large Language Models
  4. 2025PLAGUE: Plug-and-play framework for Lifelong Adaptive Generation of Multi-turn Exploits
  5. 2026MADBench: Benchmarking the Security of Multi-Agent Debate
  6. 2025Chain-of-Lure: A Universal Jailbreak Attack Framework using Unconstrained Synthetic Narratives
  7. 2026On Understanding, Identifying, and Mitigating Vulnerabilities in Agentic Large Language Models
  8. 2026A Self-Evolving Multi-Agent Framework Defense against LLM Jailbreak Attacks