mlsanityData to 7 Oct 2026

Speak Out of Turn: Safety Vulnerability of Large Language Models in Multi-turn Dialogue

arXiv 2402.17262 ↗First posted 27 Feb 2024cs.CL cs.AIcited by 64

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

Large Language Models (LLMs) have been demonstrated to generate illegal or unethical responses, particularly when subjected to "jailbreak." Research on jailbreak has highlighted the safety issues of LLMs. However, prior studies have predominantly focused on single-turn dialogue, ignoring the potential complexities and risks presented by multi-turn dialogue, a crucial mode through which humans derive information from LLMs. In this paper, we argue that humans could exploit multi-turn dialogue to induce LLMs into generating harmful information. LLMs may not intend to reject cautionary or borderline unsafe queries, even if each turn is closely served for one malicious purpose in a multi-turn dialogue. Therefore, by decomposing an unsafe query into several sub-queries for multi-turn dialogue, we induced LLMs to answer harmful sub-questions incrementally, culminating in an overall harmful response. Our experiments, conducted across a wide range of LLMs, indicate current inadequacies in the safety mechanisms of LLMs in multi-turn dialogue. Our findings expose vulnerabilities of LLMs in complex scenarios involving multi-turn dialogue, presenting new challenges for the safety of LLMs.

Key facts. Cited by 64 papers in the map (data through 7 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 24 of its 34 references that are in the map. First posted on arXiv 27 Feb 2024.

Builds on

24 of 34 references.

Closest ancestors · 6

Same direction, newest first

  1. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreaker6 Nov
  2. 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
  3. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generation10 Oct
  4. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations10 Oct
  5. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  6. 2023Jailbroken: How Does LLM Safety Training Fail?5 Jul

Other references · 18

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  7. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  8. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  9. 2020Learning to summarize from human feedbackcited by 2,562
  10. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  11. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  12. 2021Ethical and social risks of harm from Language Modelscited by 957
6 more
  1. 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 615
  2. 2023Prompt Injection attack against LLM-integrated Applicationscited by 568
  3. 2023RAIN: Your Language Models Can Align Themselves without Finetuningcited by 149
  4. 2023Question Decomposition Improves the Faithfulness of Model-Generated Reasoningcited by 107
  5. 2023Leveraging Large Language Models in Conversational Recommender Systemscited by 63
  6. 2023Are LLMs All You Need for Task-Oriented Dialogue?cited by 52

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 9; 2025: 39; 2026: 16.

64 papers.

Sort:

By area:

  1. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  2. 2024LLMs know their vulnerabilities: Uncover Safety Gaps through Natural Distribution Shiftscited by 80
  3. 2025Reasoning-Augmented Conversation for Multi-Turn Jailbreak Attacks on Large Language Modelscited by 61
  4. 2024Multi-Turn Context Jailbreak Attack on Large Language Models From First Principlescited by 54
  5. 2025Foot-In-The-Door: A Multi-turn Jailbreak for LLMscited by 50
  6. 2024FairMT-Bench: Benchmarking Fairness for Multi-turn Dialogue in Conversational LLMscited by 36
  7. 2024Jailbreaking and Mitigation of Vulnerabilities in Large Language Modelscited by 33
  8. 2025MTSA: Multi-turn Safety Alignment for LLMs through Multi-round Red-teamingcited by 31
  9. 2024MRJ-Agent: An Effective Jailbreak Agent for Multi-Round Dialoguecited by 28
  10. 2026Unsafer in Many Turns: Benchmarking and Defending Multi-Turn Safety Risks in Tool-Using Agentscited by 28
  11. 2025Jailbreaking Large Language Diffusion Models: Revealing Hidden Safety Flaws in Diffusion-Based Text Generationcited by 24
  12. 2025SafeDialBench: A Fine-Grained Safety Evaluation Benchmark for Large Language Models in Multi-Turn Dialogues with Diverse Jailbreak Attackscited by 23
38 more
  1. 2025Visual Contextual Attack: Jailbreaking MLLMs with Image-Driven Context Injectioncited by 23
  2. 2024From LLMs to MLLMs: Exploring the Landscape of Multimodal Jailbreakingcited by 22
  3. 2025X-Boundary: Establishing Exact Safety Boundary to Shield LLMs from Multi-Turn Jailbreaks without Compromising Usabilitycited by 20
  4. 2025Steering Dialogue Dynamics for Robustness against Multi-turn Jailbreaking Attackscited by 20
  5. 2025SoK: Evaluating Jailbreak Guardrails for Large Language Modelscited by 17
  6. 2024Emerging Vulnerabilities in Frontier Models: Multi-Turn Jailbreak Attackscited by 16
  7. 2025Multi-Turn Jailbreaks Are Simpler Than They Seemcited by 15
  8. 2025Siren: A Learning-Based Multi-Turn Attack Framework for Simulating Real-World Human Jailbreak Behaviorscited by 12
  9. 2025Large Reasoning Models Are Autonomous Jailbreak Agentscited by 12
  10. 2025Enhancing Model Defense Against Jailbreaks with Proactive Safety Reasoningcited by 11
  11. 2025Building Safe GenAI Applications: An End-to-End Overview of Red Teaming for Large Language Modelscited by 11
  12. 2025DiffuGuard: How Intrinsic Safety is Lost and Found in Diffusion Large Language Modelscited by 11
  13. 2025Response Attack: Exploiting Contextual Priming to Jailbreak Large Language Modelscited by 10
  14. 2024How Are LLMs Mitigating Stereotyping Harms? Learning from Search Engine Studiescited by 9
  15. 2025Exploring Jailbreak Attacks on LLMs through Intent Concealment and Diversioncited by 9
  16. 2025MIRAGE: Multimodal Immersive Reasoning and Guided Exploration for Red-Team Jailbreak Attackscited by 8
  17. 2025Does Chain-of-Thought Reasoning Really Reduce Harmfulness from Jailbreaking?cited by 8
  18. 2025Guardians and Offenders: A Survey on Harmful Content Generation and Safety Mitigation of LLMcited by 8
  19. 2025ORFuzz: Fuzzing the "Other Side" of LLM Safety -- Testing Over-Refusalcited by 6
  20. 2026Knowledge-Driven Multi-Turn Jailbreaking on Large Language Modelscited by 6
  21. 2025Dagger Behind Smile: Fool LLMs with a Happy Ending Storycited by 5
  22. 2025CoBia: Constructed Conversations Can Trigger Otherwise Concealed Societal Biases in LLMscited by 5
  23. 2025Temporal Context Awareness: A Defense Framework Against Multi-turn Manipulation Attacks on Large Language Modelscited by 4
  24. 2025OpenDeception: Learning Deception and Trust in Human-AI Interaction via Multi-Agent Simulationcited by 4
  25. 2025When Memory Becomes a Vulnerability: Towards Multi-turn Jailbreak Attacks against Text-to-Image Generation Systemscited by 4
  26. 2025REVEAL: Multi-turn Evaluation of Image-Input Harms for Vision LLMcited by 4
  27. 2025ScamAgents: How AI Agents Can Simulate Human-Level Scam Callscited by 4
  28. 2025Eliciting Behaviors in Multi-Turn Conversationscited by 4
  29. 2026State-Dependent Safety Failures in Multi-Turn Language Model Interactioncited by 4
  30. 2024Exploring Straightforward Conversational Red-Teamingcited by 3
  31. 2025Many-Turn Jailbreakingcited by 2
  32. 2025How Catastrophic is Your LLM? Certifying Risk in Conversationcited by 2
  33. 2025VRSA: Jailbreaking Multimodal Large Language Models through Visual Reasoning Sequential Attackcited by 2
  34. 2025Breaking Minds, Breaking Systems: Jailbreaking Large Language Models via Human-like Psychological Manipulationcited by 2
  35. 2026When Choices Become Risks: Safety Failures of Large Language Models under Multiple-Choice Constraintscited by 2
  36. 2025PD3F: A Pluggable and Dynamic DoS-Defense Framework Against Resource Consumption Attacks Targeting Large Language Modelscited by 1
  37. 2025Layer-Wise Perturbations via Sparse Autoencoders for Adversarial Text Generationcited by 1
  38. 2025LLaVAShield: Safeguarding Multimodal Multi-Turn Dialogues in Vision-Language Modelscited by 1

50 most cited of 64 All 64 (JSON)