mlsanityData to 7 Oct 2026

LLM Defenses Are Not Robust to Multi-Turn Human Jailbreaks Yet

arXiv 2408.15221 ↗First posted 27 Aug 2024cs.LG cs.CL cs.CRcited by 145

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

Recent large language model (LLM) defenses have greatly improved models' ability to refuse harmful queries, even when adversarially attacked. However, LLM defenses are primarily evaluated against automated adversarial attacks in a single turn of conversation, an insufficient threat model for real-world malicious use. We demonstrate that multi-turn human jailbreaks uncover significant vulnerabilities, exceeding 70% attack success rate (ASR) on HarmBench against defenses that report single-digit ASRs with automated single-turn attacks. Human jailbreaks also reveal vulnerabilities in machine unlearning defenses, successfully recovering dual-use biosecurity knowledge from unlearned models. We compile these results into Multi-Turn Human Jailbreaks (MHJ), a dataset of 2,912 prompts across 537 multi-turn jailbreaks. We publicly release MHJ alongside a compendium of jailbreak tactics developed across dozens of commercial red teaming engagements, supporting research towards stronger LLM defenses.

Key facts. Cited by 145 papers in the map (data through 7 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 61 of its 118 references that are in the map. First posted on arXiv 27 Aug 2024.

Builds on

61 of 118 references.

Closest ancestors · 20

Same direction, newest first

  1. 2024Multi-Turn Context Jailbreak Attack on Large Language Models From First Principles8 Aug
  2. 2024Tamper-Resistant Safeguards for Open-Weight LLMs1 Aug
  3. 2024Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMs22 Jul
  4. 2024WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Models26 Jun
  5. 2024Efficient Adversarial Training in LLMs with Continuous Attacks24 May
  6. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks2 Apr
  7. 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attack2 Apr
  8. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models28 Mar
  9. 2024DrAttack: Prompt Decomposition and Reconstruction Makes Powerful LLM Jailbreakers25 Feb
  10. 2024PRP: Propagating Universal Perturbations to Attack Large Language Model Guard-Rails24 Feb
  11. 2024When "Competency" in Reasoning Opens the Door to Vulnerability: Jailbreaking LLMs via Novel Complex Ciphers16 Feb
  12. 2024Attacking Large Language Models with Projected Gradient Descent14 Feb
8 more
  1. 2024Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attacks30 Jan
  2. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs12 Jan
  3. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automatically4 Dec
  4. 2023A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily14 Nov
  5. 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
  6. 2023Low-Resource Languages Jailbreak GPT-43 Oct
  7. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts19 Sep
  8. 2023Jailbroken: How Does LLM Safety Training Fail?5 Jul

Other references · 41

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  5. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  6. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  7. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  8. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  9. 2016Adversarial examples in the physical worldcited by 1,762
  10. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  11. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
  12. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
29 more
  1. 2019Machine Unlearningcited by 1,020
  2. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
  3. 2022Red Teaming Language Models with Language Modelscited by 983
  4. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  5. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 950
  6. 2023Steering Language Models With Activation Engineeringcited by 824
  7. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
  8. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  9. 2017Adversarial Examples for Evaluating Reading Comprehension Systemscited by 568
  10. 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deepcited by 450
  11. 2023Are aligned neural networks adversarially aligned?cited by 305
  12. 2024Improving Alignment and Robustness with Circuit Breakerscited by 302
  13. 2021Unsolved Problems in ML Safetycited by 291
  14. 2024ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMscited by 197
  15. 2023LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70Bcited by 183
  16. 2023MART: Improving LLM Safety with Multi-round Automatic Red-Teamingcited by 161
  17. 2024Eight Methods to Evaluate Robust Unlearning in LLMscited by 153
  18. 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 123
  19. 2024Large Language Model Unlearning via Embedding-Corrupted Promptscited by 112
  20. 2023Explore, Establish, Exploit: Red Teaming Language Models from Scratchcited by 111
  21. 2024Representation Noising: A Defence Mechanism Against Harmful Finetuningcited by 93
  22. 2024Covert Malicious Finetuning: Challenges in Safeguarding LLM Adaptationcited by 74
  23. 2024Does Refusal Training in LLMs Generalize to the Past Tense?cited by 74
  24. 2024Feedback Loops With Language Models Drive In-Context Reward Hackingcited by 70
  25. 2024Refuse Whenever You Feel Unsafe: Improving Safety in LLMs via Decoupled Refusal Trainingcited by 67
  26. 2023LLM Censorship: A Machine Learning Challenge or a Computer Security Problem?cited by 47
  27. 2024PAL: Proxy-Guided Black-Box Attack on Large Language Modelscited by 46
  28. 2023Summon a Demon and Bind it: A Grounded Theory of LLM Red Teamingcited by 20
  29. 2024Revisiting the Robust Alignment of Circuit Breakerscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 15; 2025: 80; 2026: 50.

145 papers.

Sort:

By area:

  1. 2024AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agentscited by 406
  2. 2025The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injectionscited by 108
  3. 2025The Hidden Risks of Large Reasoning Models: A Safety Assessment of R1cited by 89
  4. 2024LLMs know their vulnerabilities: Uncover Safety Gaps through Natural Distribution Shiftscited by 80
  5. 2024Do Unlearning Methods Remove Information from Language Model Weights?cited by 66
  6. 2024Refusal-Trained LLMs Are Easily Jailbroken As Browser Agentscited by 61
  7. 2024Position: LLM Unlearning Benchmarks are Weak Measures of Progresscited by 58
  8. 2025Safety in Large Reasoning Models: A Surveycited by 58
  9. 2025Foot-In-The-Door: A Multi-turn Jailbreak for LLMscited by 50
  10. 2024Automated Red Teaming with GOAT: the Generative Offensive Agent Testercited by 41
  11. 2025AutoRedTeamer: Autonomous Red Teaming with Lifelong Attack Integrationcited by 38
  12. 2024FairMT-Bench: Benchmarking Fairness for Multi-turn Dialogue in Conversational LLMscited by 36
38 more
  1. 2025Model Tampering Attacks Enable More Rigorous Evaluations of LLM Capabilitiescited by 36
  2. 2025MTSA: Multi-turn Safety Alignment for LLMs through Multi-round Red-teamingcited by 31
  3. 2025Adversarial Attacks on Robotic Vision Language Action Modelscited by 31
  4. 2026AgentLAB: Benchmarking LLM Agents against Long-Horizon Attackscited by 31
  5. 2025Safety Guardrails for LLM-Enabled Robotscited by 30
  6. 2025ChatInject: Abusing Chat Templates for Prompt Injection in LLM Agentscited by 28
  7. 2026Unsafer in Many Turns: Benchmarking and Defending Multi-Turn Safety Risks in Tool-Using Agentscited by 28
  8. 2025Persuade Me if You Can: A Framework for Evaluating Persuasion Effectiveness and Susceptibility Among Large Language Modelscited by 25
  9. 2025Accumulating Context Changes the Beliefs of Language Modelscited by 25
  10. 2025Reliable Weak-to-Strong Monitoring of LLM Agentscited by 24
  11. 2024Locking Down the Finetuned LLMs Safetycited by 22
  12. 2025Security Challenges in AI Agent Deployment: Insights from a Large Scale Public Competitioncited by 22
  13. 2025Eliciting Language Model Behaviors with Investigator Agentscited by 21
  14. 2025X-Boundary: Establishing Exact Safety Boundary to Shield LLMs from Multi-Turn Jailbreaks without Compromising Usabilitycited by 20
  15. 2025Steering Dialogue Dynamics for Robustness against Multi-turn Jailbreaking Attackscited by 20
  16. 2025From Judgment to Interference: Early Stopping LLM Harmful Outputs via Streaming Content Monitoringcited by 20
  17. 2025FORTRESS: Frontier Risk Evaluation for National Security and Public Safetycited by 20
  18. 2025RL-Obfuscation: Can Language Models Learn to Evade Latent-Space Monitors?cited by 19
  19. 2024Endless Jailbreaks with Bijection Learningcited by 18
  20. 2025Reality Check: A New Evaluation Ecosystem Is Necessary to Understand AI's Real World Effectscited by 18
  21. 2026OpenRT: An Open-Source Red Teaming Framework for Multimodal LLMscited by 18
  22. 2025SoK: Evaluating Jailbreak Guardrails for Large Language Modelscited by 17
  23. 2024Emerging Vulnerabilities in Frontier Models: Multi-Turn Jailbreak Attackscited by 16
  24. 2025The Elicitation Game: Evaluating Capability Elicitation Techniquescited by 15
  25. 2025Tempest: Autonomous Multi-Turn Jailbreaking of Large Language Models with Tree Searchcited by 15
  26. 2025Position: Adversarial ML for LLMs Is Not Making Any Progresscited by 14
  27. 2025Speak Easy: Eliciting Harmful Jailbreaks from LLMs with Simple Interactionscited by 14
  28. 2025Think Twice Before You Act: Enhancing Agent Behavioral Safety with Thought Correctioncited by 14
  29. 2025Siren: A Learning-Based Multi-Turn Attack Framework for Simulating Real-World Human Jailbreak Behaviorscited by 12
  30. 2025Must Read: A Comprehensive Survey of Computational Persuasioncited by 12
  31. 2025Building Safe GenAI Applications: An End-to-End Overview of Red Teaming for Large Language Modelscited by 11
  32. 2025TROJail: Trajectory-Level Optimization for Multi-Turn Large Language Model Jailbreaks with Process Rewardscited by 11
  33. 2025Transferable Adversarial Attacks on Black-Box Vision-Language Modelscited by 10
  34. 2025A Representation Engineering Perspective on the Effectiveness of Multi-Turn Jailbreakscited by 10
  35. 2026SEMA: Simple yet Effective Learning for Multi-Turn Jailbreak Attackscited by 10
  36. 2025Using Mechanistic Interpretability to Craft Adversarial Attacks against Large Language Modelscited by 9
  37. 2025Jailbreaking to Jailbreakcited by 8
  38. 2025Adversarial Alignment for LLMs Requires Simpler, Reproducible, and More Measurable Objectivescited by 8

50 most cited of 145 All 145 (JSON)