mlsanityData to 7 Oct 2026

Adversarial Attack-Defense Co-Evolution for LLM Safety Alignment via Tree-Group Dual-Aware Search and Optimization

arXiv 2511.19218 ↗First posted 24 Nov 2025cs.CR cs.AIcited by 4

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

also in 1 more
Abstract (arXiv)

Large Language Models (LLMs) have developed rapidly in web services, delivering unprecedented capabilities while amplifying societal risks. Existing works tend to focus on either isolated jailbreak attacks or static defenses, neglecting the dynamic interplay between evolving threats and safeguards in real-world web contexts. To mitigate these challenges, we propose ACE-Safety (Adversarial Co-Evolution for LLM Safety), a novel framework that jointly optimize attack and defense models by seamlessly integrating two key innovative procedures: (1) Group-aware Strategy-guided Monte Carlo Tree Search (GS-MCTS), which efficiently explores jailbreak strategies to uncover vulnerabilities and generate diverse adversarial samples; (2) Adversarial Curriculum Tree-aware Group Policy Optimization (AC-TGPO), which jointly trains attack and defense LLMs with challenging samples via curriculum reinforcement learning, enabling robust mutual improvement. Evaluations across multiple benchmarks demonstrate that our method outperforms existing attack and defense approaches, and provides a feasible pathway for developing LLMs that can sustainably support responsible AI ecosystems.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 32 of its 53 references that are in the map. First posted on arXiv 24 Nov 2025.

Builds on

32 of 53 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025Lifelong Safety Alignment for Language Models26 May
  2. 2025PANDAS: Improving Many-shot Jailbreaking via Positive Affirmation, Negative Demonstration, and Adaptive Sampling4 Feb
  3. 2024Iterative Self-Tuning LLMs for Enhanced Jailbreaking Capabilities24 Oct
  4. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedback19 Oct
  5. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!5 Oct
  6. 2023Defending Against Alignment-Breaking Attacks via Robustly Aligned LLM18 Sep
  7. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul

Other references · 25

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  5. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  6. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  7. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  8. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 738
  9. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 520
  10. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 478
  11. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 434
  12. 2023Detecting Language Model Attacks with Perplexitycited by 411
13 more
  1. 2024Improving Alignment and Robustness with Circuit Breakerscited by 302
  2. 2023Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attackscited by 268
  3. 2023A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easilycited by 242
  4. 2024SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decodingcited by 233
  5. 2023ToxicChat: Unveiling Hidden Challenges of Toxicity Detection in Real-World User-AI Conversationcited by 224
  6. 2023MART: Improving LLM Safety with Multi-round Automatic Red-Teamingcited by 161
  7. 2024Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMscited by 158
  8. 2024Efficient Adversarial Training in LLMs with Continuous Attackscited by 104
  9. 2023CValues: Measuring the Values of Chinese Large Language Models from Safety to Responsibilitycited by 80
  10. 2024Improved Few-Shot Jailbreaking Can Circumvent Aligned Language Models and Their Defensescited by 68
  11. 2024Towards Understanding Jailbreak Attacks in LLMs: A Representation Space Analysiscited by 61
  12. 2022Order-Disorder: Imitation Adversarial Attacks for Black-box Neural Ranking Modelscited by 37
  13. 2024Defending Jailbreak Prompts via In-Context Adversarial Gamecited by 27

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026MAGIC: A Co-Evolving Attacker-Defender Adversarial Game for Robust LLM Safetycited by 15
  2. 2026Be Your Own Red Teamer: Safety Alignment via Self-Play and Reflective Experience Replaycited by 6
  3. 2026How to Compress KV Cache in RL Post-Training? Shadow Mask Distillation for Memory-Efficient Alignmentcited by 2
  4. 2026Co-Evolution in Agentic Systems: Toward Self-Directed Evolution Beyond Human Designcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026Safety Is Not Universal: The Selective Safety Trap in LLM Alignment
  2. 2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attacks
  3. 2026Metis: Learning to Jailbreak LLMs via Self-Evolving Metacognitive Policy Optimization
  4. 2025Adversarial Reinforcement Learning for Large Language Model Agent Safety
  5. 2026CHASE: Adversarial Red-Blue Teaming for Improving LLM Safety using Reinforcement Learning
  6. 2026Co-Evolutionary Multi-Modal Alignment via Structured Adversarial Evolution
  7. 2025MTSA: Multi-turn Safety Alignment for LLMs through Multi-round Red-teaming
  8. 2026CoopGuard: Stateful Cooperative Agents Safeguarding LLMs Against Evolving Multi-Round Attacks