mlsanityData to 14 Oct 2026

Reasoning-Augmented Conversation for Multi-Turn Jailbreak Attacks on Large Language Models

arXiv 2502.11054 ↗First posted 16 Feb 2025cs.CL cs.AI cs.CRcited by 64

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 1 more
Abstract (arXiv)

Multi-turn jailbreak attacks simulate real-world human interactions by engaging large language models (LLMs) in iterative dialogues, exposing critical safety vulnerabilities. However, existing methods often struggle to balance semantic coherence with attack effectiveness, resulting in either benign semantic drift or ineffective detection evasion. To address this challenge, we propose Reasoning-Augmented Conversation, a novel multi-turn jailbreak framework that reformulates harmful queries into benign reasoning tasks and leverages LLMs' strong reasoning capabilities to compromise safety alignment. Specifically, we introduce an attack state machine framework to systematically model problem translation and iterative reasoning, ensuring coherent query generation across multiple turns. Building on this framework, we design gain-guided exploration, self-play, and rejection feedback modules to preserve attack semantics, enhance effectiveness, and sustain reasoning-driven attack progression. Extensive experiments on multiple LLMs demonstrate that RACE achieves state-of-the-art attack effectiveness in complex conversational scenarios, with attack success rates (ASRs) increasing by up to 96%. Notably, our approach achieves ASRs of 82% and 92% against leading commercial models, OpenAI o1 and DeepSeek R1, underscoring its potency. We release our code at https://github.com/NY1024/RACE to facilitate further research in this critical domain.

Key facts. Cited by 64 papers in the map (data through 14 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 43 of its 69 references that are in the map. First posted on arXiv 16 Feb 2025.

Builds on

43 of 69 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs12 Jan

Other references · 42

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  4. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  5. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  6. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  7. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
  8. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,407
  9. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,121
  10. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 785
  11. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 672
  12. 2024LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methodscited by 465
30 more
  1. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrationscited by 431
  2. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreakercited by 367
  3. 2023Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspectivecited by 294
  4. 2023Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Futurecited by 184
  5. 2022Talking About Large Language Modelscited by 175
  6. 2022Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanationscited by 159
  7. 2023Open Sesame! Universal Black Box Jailbreaking of Large Language Modelscited by 134
  8. 2023BadCLIP: Dual-Embedding Guided Backdoor Attack on Multimodal Contrastive Learningcited by 115
  9. 2024Jailbreak Vision Language Models via Bi-Modal Adversarial Promptcited by 109
  10. 2023CLadder: Assessing Causal Reasoning in Language Modelscited by 107
  11. 2024A Comprehensive Study of Jailbreak Attack versus Defense for Large Language Modelscited by 106
  12. 2024VL-Trojan: Multimodal Instruction Backdoor Attacks against Autoregressive Visual Language Modelscited by 87
  13. 2024LLMs know their vulnerabilities: Uncover Safety Gaps through Natural Distribution Shiftscited by 86
  14. 2024PsySafe: A Comprehensive Framework for Psychological-based Attack, Defense, and Evaluation of Multi-agent System Safetycited by 80
  15. 2024Chain of Attack: a Semantic-Driven Contextual Multi-Turn attacker for LLMcited by 77
  16. 2024SafeBench: A Safety Evaluation Framework for Multimodal Large Language Modelscited by 69
  17. 2024Speak Out of Turn: Safety Vulnerability of Large Language Models in Multi-turn Dialoguecited by 66
  18. 2024Multi-Turn Context Jailbreak Attack on Large Language Models From First Principlescited by 56
  19. 2024Semantic Mirror Jailbreak: Genetic Algorithm Based Jailbreak Prompts Against Open-source LLMscited by 53
  20. 2023JailGuard: A Universal Detection Framework for LLM Prompt-based Attackscited by 50
  21. 2024Unveiling the Safety of GPT-4o: An Empirical Study using Jailbreak Attackscited by 38
  22. 2024Leveraging the Context through Multi-Round Interactions for Jailbreaking Attackscited by 36
  23. 2024Visual Adversarial Attack on Vision-Language Models for Autonomous Drivingcited by 36
  24. 2024Foot In The Door: Understanding Large Language Model Jailbreaking via Cognitive Psychologycited by 27
  25. 2025Black-Box Adversarial Attack on Vision Language Models for Autonomous Drivingcited by 24
  26. 2022Exploring the Relationship between Architecture and Adversarially Robust Generalizationcited by 23
  27. 2024LanEvil: Benchmarking the Robustness of Lane Detection to Environmental Illusionscited by 18
  28. 2024Uncertainty is Fragile: Manipulating Uncertainty in Large Language Modelscited by 13
  29. 2024DLP: towards active defense against backdoor attacks with decoupled learning processcited by 10
  30. 2024NBA: defensive distillation for backdoor removal via neural behavior alignmentcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 39; 2026: 25.

64 papers.

Sort:

By area:

  1. 2025X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agentscited by 125
  2. 2025Breaking the Exploration Bottleneck: Rubric-Scaffolded Reinforcement Learning for General LLM Reasoningcited by 63
  3. 2025Manipulating Multimodal Agents via Cross-Modal Prompt Injectioncited by 44
  4. 2025AGENTSAFE: Benchmarking the Safety of Embodied Agents on Hazardous Instructionscited by 38
  5. 2025Towards Understanding the Safety Boundaries of DeepSeek Models: Evaluation and Findingscited by 27
  6. 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Modelscited by 25
  7. 2025T2V-OptJail: Discrete Prompt Optimization for Text-to-Video Jailbreak Attackscited by 21
  8. 2026OpenRT: An Open-Source Red Teaming Framework for Multimodal LLMscited by 21
  9. 2026MAGIC: A Co-Evolving Attacker-Defender Adversarial Game for Robust LLM Safetycited by 21
  10. 2026Uncovering Security Threats and Architecting Defenses in Autonomous Agents: A Case Study of OpenClawcited by 20
  11. 2025Multi-Turn Jailbreaks Are Simpler Than They Seemcited by 17
  12. 2025A Survey of Attacks on Large Language Modelscited by 15
38 more
  1. 2025SafeMobile: Chain-level Jailbreak Detection and Automated Evaluation for Multimodal Mobile Agentscited by 15
  2. 2025Steering the CensorShip: Uncovering Representation Vectors for LLM "Thought" Controlcited by 14
  3. 2025Large Reasoning Models Are Autonomous Jailbreak Agentscited by 13
  4. 2025E2AT: Multimodal Jailbreak Defense via Dynamic Joint Optimization for Multimodal Large Language Modelscited by 12
  5. 2025Pushing the Limits of Safety: A Technical Report on the ATLAS Challenge 2025cited by 12
  6. 2025Detoxifying Large Language Models via Autoregressive Reward Guided Representation Editingcited by 10
  7. 2025Three Minds, One Legend: Jailbreak Large Reasoning Model with Adaptive Stacked Cipherscited by 8
  8. 2025Tree-based Dialogue Reinforced Policy Optimization for Red-Teaming Attackscited by 8
  9. 2025Evolve the Method, Not the Prompts: Evolutionary Synthesis of Jailbreak Attacks on LLMscited by 8
  10. 2026DeepContext: Stateful Real-Time Detection of Multi-Turn Adversarial Intent Drift in LLMscited by 8
  11. 2025Proactive defense against LLM Jailbreakcited by 7
  12. 2026What Matters For Safety Alignment?cited by 7
  13. 2026Knowledge-Driven Multi-Turn Jailbreaking on Large Language Modelscited by 7
  14. 2026CtrlAttack: A Unified Attack on World-Model Control in Diffusion Modelscited by 7
  15. 2025POISONCRAFT: Practical Poisoning of Retrieval-Augmented Generation for Large Language Modelscited by 6
  16. 2025Mask-GCG: Are All Tokens in Adversarial Suffixes Necessary for Jailbreak Attacks?cited by 6
  17. 2025Path Drift in Large Reasoning Models:How First-Person Commitments Override Safetycited by 5
  18. 2025Strong but Brittle: Simple Attacks Subvert Reasoning-based Safety Guardrailscited by 5
  19. 2025VRSA: Jailbreaking Multimodal Large Language Models through Visual Reasoning Sequential Attackcited by 5
  20. 2025HarmNet: A Framework for Adaptive Multi-Turn Jailbreak Attacks on Large Language Modelscited by 4
  21. 2025SPARK: Jailbreaking T2V Models by Synergistically Prompting Auditory and Recontextualized Knowledgecited by 4
  22. 2026Just Ask: Curious Code Agents Reveal System Prompts in Frontier LLMscited by 4
  23. 2026State-Dependent Safety Failures in Multi-Turn Language Model Interactioncited by 4
  24. 2025DeepSeek: Paradigm Shifts and Technical Evolution in Large AI Modelscited by 3
  25. 2025PRISM: Programmatic Reasoning with Image Sequence Manipulation for LVLM Jailbreakingcited by 3
  26. 2025PLAGUE: Plug-and-play framework for Lifelong Adaptive Generation of Multi-turn Exploitscited by 3
  27. 2026Do Reasoning LLMs Refuse What They Infer in Long Contexts?cited by 3
  28. 2026MT-JailBench: A Modular Benchmark for Understanding Multi-Turn Jailbreak Attackscited by 3
  29. 2025PRJ: Perception-Retrieval-Judgement for Generated Imagescited by 2
  30. 2025PromptSafe: Gated Prompt Tuning for Safe Text-to-Image Generationcited by 2
  31. 2025Jailbreaking LLMs via Semantically Relevant Nested Scenarios with Targeted Toxic Knowledgecited by 2
  32. 2025Sequential Comics for Jailbreaking Multimodal Large Language Models via Structured Visual Storytellingcited by 2
  33. 2025Breaking Minds, Breaking Systems: Jailbreaking Large Language Models via Human-like Psychological Manipulationcited by 2
  34. 2025SoK: A Comprehensive Security Analysis of Jailbreak Resilience in GPT and DeepSeek Modelscited by 1
  35. 2025LLaVAShield: Safeguarding Multimodal Multi-Turn Dialogues in Vision-Language Modelscited by 1
  36. 2025Bits Leaked per Query: Information-Theoretic Bounds on Adversarial Attacks against LLMscited by 1
  37. 2026Break Me If You Can: Self-Jailbreaking of Aligned LLMs via Lexical Insertion Promptingcited by 1
  38. 2026Improving Safety Alignment via Balanced Direct Preference Optimizationcited by 1

50 most cited of 64 All 64 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Reasoning-to-Defend: Safety-Aware Reasoning Can Defend Large Language Models from Jailbreaking
  2. 2024Reasoning as a Weapon: Adaptive Dual-Path Jailbreak Attack on Large Language Models
  3. 2025Pattern Enhanced Multi-Turn Jailbreaking: Exploiting Structural Vulnerabilities in Large Language Models
  4. 2025GRAF: Multi-turn Jailbreaking via Global Refinement and Active Fabrication
  5. 2025MUSE: MCTS-Driven Red Teaming Framework for Enhanced Multi-Turn Dialogue Safety in Large Language Models
  6. 2025Automating Deception: Scalable Multi-Turn LLM Jailbreaks
  7. 2026THRD: A Training-Free Multi-Turn Defense Framework for Jailbreak Attacks on Large Language Models
  8. 2026TRACE: Trajectory Aware Reasoning for Multi-Turn Adversarial Conversation Evaluation