mlsanityData to 7 Oct 2026

Stateful Cooperative Agents Safeguarding LLMs Against Evolving Multi-Turn Attacks

arXiv 2608.00134 ↗First posted 31 Jul 2026cs.CRcited by 0

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

As LLMs become increasingly integrated into complex applications, their vulnerability to adversarial attacks has raised significant concerns. However, existing defenses remain reactive in nature. This limitation makes it difficult for them to counter sophisticated threats, as adversaries continuously adjust their strategies across multi-turn interactions. In this paper, we present a proactive defense framework for securing LLMs against evolving multi-turn adversarial attacks that combines disruption, misdirection, and adaptation across successive interaction turns. In particular, it employs a cooperative multi-agent architecture in which specialized agents execute complementary defense strategies. These strategies include controlled response pacing to increase attack costs, strategically ambiguous outputs to mislead adversaries into ineffective strategies, and forensic analysis of interaction logs to identify attack patterns and refine defenses. These agents are coordinated by an adaptive mechanism that dynamically adjusts the defense strategy in response to escalating threats. To facilitate comprehensive evaluation, we present the EMRA dataset designed to simulate evolving strategies across multi-turn attacks, including 5,200 adversarial samples across eight attack types. Experimental results on EMRA across multiple LLM backbones show that the proposed framework reduces ASR by 69% on average relative to evaluated state-of-the-art baselines. Beyond suppressing harmful outputs, it sustains deceptive engagement, achieving an average DR more than six times that of the strongest baselines and increasing attacker-token consumption by 198.83% on average relative to evaluated baselines. Code and dataset are available at https://github.com/SiyuanLi00/CoopGuard.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 46 of its 66 references that are in the map. First posted on arXiv 31 Jul 2026.

Builds on

46 of 66 references.

Closest ancestors · 3

Same direction, newest first

  1. 2026HoneyTrap: Deceiving Large Language Model Attackers to Honeypot Traps with Resilient Multi-Agent Defense7 Jan
  2. 2025X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agents15 Apr
  3. 2025A Survey on Trustworthy LLM Agents: Threats and Countermeasures12 Mar

Other references · 43

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  4. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  5. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  6. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  7. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 995
  8. 2023ChatDev: Communicative Agents for Software Developmentcited by 893
  9. 2023Gemini: A Family of Highly Capable Multimodal Modelscited by 810
  10. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 738
  11. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
  12. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 533
31 more
  1. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 520
  2. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 478
  3. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generationcited by 439
  4. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 434
  5. 2023Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructionscited by 377
  6. 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 364
  7. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreakercited by 337
  8. 2023Certifying LLM Safety against Adversarial Promptingcited by 248
  9. 2024MT-Bench-101: A Fine-Grained Benchmark for Evaluating Large Language Models in Multi-Turn Dialoguescited by 247
  10. 2023Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignmentcited by 213
  11. 2023Defending Against Alignment-Breaking Attacks via Robustly Aligned LLMcited by 173
  12. 2023MasterKey: Automated Jailbreak Across Multiple Large Language Model Chatbotscited by 172
  13. 2024AdvPrompter: Fast Adaptive Adversarial Prompting for LLMscited by 146
  14. 2025DataSentinel: A Game-Theoretic Detection of Prompt Injection Attackscited by 124
  15. 2023A Survey of Large Language Models in Medicine: Progress, Application, and Challengecited by 123
  16. 2024Attacking Large Language Models with Projected Gradient Descentcited by 91
  17. 2023Training Socially Aligned Language Models on Simulated Social Interactionscited by 77
  18. 2024JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Modelscited by 67
  19. 2024Defending LLMs against Jailbreaking Attacks via Backtranslationcited by 65
  20. 2024Bag of Tricks: Benchmarking of Jailbreak Attacks on LLMscited by 47
  21. 2024From Theft to Bomb-Making: The Ripple Effect of Unlearning in Defending Against Jailbreak Attackscited by 29
  22. 2024PAPILLON: Efficient and Stealthy Fuzz Testing-Powered Jailbreaks for LLMscited by 20
  23. 2025X-Boundary: Establishing Exact Safety Boundary to Shield LLMs from Multi-Turn Jailbreaks without Compromising Usabilitycited by 20
  24. 2025Steering Dialogue Dynamics for Robustness against Multi-turn Jailbreaking Attackscited by 20
  25. 2024Exploiting the Index Gradients for Optimization-Based Jailbreaking on Large Language Modelscited by 15
  26. 2025Tempest: Autonomous Multi-Turn Jailbreaking of Large Language Models with Tree Searchcited by 15
  27. 2023Exploring Recommendation Capabilities of GPT-4V(ision): A Preliminary Case Studycited by 12
  28. 2025SecurityLingua: Efficient Defense of LLM Jailbreak Attacks via Security-Aware Prompt Compressioncited by 12
  29. 2024Boosting Jailbreak Transferability for Large Language Modelscited by 4
  30. 2024RED QUEEN: Safeguarding Large Language Models against Concealed Multi-Turn Jailbreakingcited by 2
  31. 2025Helping Large Language Models Protect Themselves: An Enhanced Filtering and Summarization Systemcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026CoopGuard: Stateful Cooperative Agents Safeguarding LLMs Against Evolving Multi-Round Attacks
  2. 2026Adaptive Adversaries: A Multi-Turn, Multi-LLM Benchmark for LLM Agent Security
  3. 2026Safety in Self-Evolving LLM Agent Systems: Threats, Amplification, and Case Studies
  4. 2026MAGIC: A Co-Evolving Attacker-Defender Adversarial Game for Robust LLM Safety
  5. 2025Adversarial Attack-Defense Co-Evolution for LLM Safety Alignment via Tree-Group Dual-Aware Search and Optimization
  6. 2025AegisLLM: Scaling Agentic Systems for Self-Reflective Defense in LLM Security
  7. 2025AdvEvo-MARL: Shaping Internalized Safety through Adversarial Co-Evolution in Multi-Agent Reinforcement Learning
  8. 2026Defending LLM-based Multi-Agent Systems Against Cooperative Attacks with Sentence-Level Rectification