mlsanityData to 14 Oct 2026

Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attack

arXiv 2404.01833 ↗First posted 2 Apr 2024cs.CR cs.AIcited by 390

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

Large Language Models (LLMs) have risen significantly in popularity and are increasingly being adopted across multiple applications. These LLMs are heavily aligned to resist engaging in illegal or unethical topics as a means to avoid contributing to responsible AI harms. However, a recent line of attacks, known as jailbreaks, seek to overcome this alignment. Intuitively, jailbreak attacks aim to narrow the gap between what the model can do and what it is willing to do. In this paper, we introduce a novel jailbreak attack called Crescendo. Unlike existing jailbreak methods, Crescendo is a simple multi-turn jailbreak that interacts with the model in a seemingly benign manner. It begins with a general prompt or question about the task at hand and then gradually escalates the dialogue by referencing the model's replies progressively leading to a successful jailbreak. We evaluate Crescendo on various public systems, including ChatGPT, Gemini Pro, Gemini-Ultra, LlaMA-2 70b and LlaMA-3 70b Chat, and Anthropic Chat. Our results demonstrate the strong efficacy of Crescendo, with it achieving high attack success rates across all evaluated models and tasks. Furthermore, we present Crescendomation, a tool that automates the Crescendo attack and demonstrate its efficacy against state-of-the-art models through our evaluations. Crescendomation surpasses other state-of-the-art jailbreaking techniques on the AdvBench subset dataset, achieving 29-61% higher performance on GPT-4 and 49-71% on Gemini-Pro. Finally, we also demonstrate Crescendo's ability to jailbreak multimodal models.

Key facts. Cited by 390 papers in the map (data through 14 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 19 of its 32 references that are in the map. First posted on arXiv 2 Apr 2024.

Builds on

19 of 32 references.

Closest ancestors · 11

Same direction, newest first

  1. 2024Leveraging the Context through Multi-Round Interactions for Jailbreaking Attacks14 Feb
  2. 2023Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritization15 Nov
  3. 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
  4. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generation10 Oct
  5. 2023Multilingual Jailbreak Challenges in Large Language Models10 Oct
  6. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models3 Oct
  7. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts19 Sep
  8. 2023"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models7 Aug
  9. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  10. 2023Jailbroken: How Does LLM Safety Training Fail?5 Jul
  11. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Study23 May

Other references · 8

  1. 2020Extracting Training Data from Large Language Modelscited by 2,189
  2. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
  3. 2022Improving alignment of dialogue agents via targeted human judgementscited by 531
  4. 2023Analyzing Leakage of Personally Identifiable Information in Language Modelscited by 285
  5. 2023Make Them Spill the Beans! Coercive Knowledge Extraction from (Production) LLMscited by 27
  6. 2023SC-Safety: A Multi-round Open-ended Question Adversarial Safety Benchmark for Large Language Models in Chinesecited by 17
  7. 2023Last One Standing: A Comparative Analysis of Security and Privacy of Soft Prompt Tuning, LoRA, and In-Context Learningcited by 14
  8. 2023Ethical Frameworks and Computer Security Trolley Problems: Foundations for Conversationscited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 47; 2025: 132; 2026: 211.

390 papers.

Sort:

By area:

  1. 2024Magentic-One: A Generalist Multi-Agent System for Solving Complex Taskscited by 232
  2. 2024LLM Defenses Are Not Robust to Multi-Turn Human Jailbreaks Yetcited by 151
  3. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 116
  4. 2025SafeArena: Evaluating the Safety of Autonomous Web Agentscited by 102
  5. 2025A Survey on Trustworthy LLM Agents: Threats and Countermeasurescited by 96
  6. 2024Steering Language Model Refusal with Sparse Autoencoderscited by 95
  7. 2024LLMs know their vulnerabilities: Uncover Safety Gaps through Natural Distribution Shiftscited by 86
  8. 2024FlipAttack: Jailbreak LLMs via Flippingcited by 80
  9. 2024Does Refusal Training in LLMs Generalize to the Past Tense?cited by 79
  10. 2024Refusal-Trained LLMs Are Easily Jailbroken As Browser Agentscited by 65
  11. 2025Safety in Large Reasoning Models: A Surveycited by 64
  12. 2024Multi-Turn Context Jailbreak Attack on Large Language Models From First Principlescited by 56
38 more
  1. 2025Foot-In-The-Door: A Multi-turn Jailbreak for LLMscited by 54
  2. 2024On Evaluating the Durability of Safeguards for Open-Weight LLMscited by 52
  3. 2024Bag of Tricks: Benchmarking of Jailbreak Attacks on LLMscited by 51
  4. 2024JAILJUDGE: A Comprehensive Jailbreak Judge Benchmark with Multi-Agent Enhanced Explanation Evaluation Frameworkcited by 47
  5. 2024Automated Red Teaming with GOAT: the Generative Offensive Agent Testercited by 45
  6. 2025Improving LLM Safety Alignment with Dual-Objective Optimizationcited by 43
  7. 2025Safety is Not Only About Refusal: Reasoning-Enhanced Fine-tuning for Interpretable LLM Safetycited by 39
  8. 2026AgentLAB: Benchmarking LLM Agents against Long-Horizon Attackscited by 39
  9. 2024Feedback-Guided Extraction of Knowledge Base from Retrieval-Augmented LLM Applicationscited by 37
  10. 2025Safety Guardrails for LLM-Enabled Robotscited by 35
  11. 2024Jailbreaking and Mitigation of Vulnerabilities in Large Language Modelscited by 34
  12. 2025Reasoning-to-Defend: Safety-Aware Reasoning Can Defend Large Language Models from Jailbreakingcited by 34
  13. 2024MRJ-Agent: An Effective Jailbreak Agent for Multi-Round Dialoguecited by 33
  14. 2025Who is Introducing the Failure? Automatically Attributing Failures of Multi-Agent Systems via Spectrum Analysiscited by 33
  15. 2024Phi-3 Safety Post-Training: Aligning Language Models with a "Break-Fix" Cyclecited by 32
  16. 2025Adversarial Attacks on Robotic Vision Language Action Modelscited by 32
  17. 2024JailbreakEval: An Integrated Toolkit for Evaluating Jailbreak Attempts Against Large Language Modelscited by 31
  18. 2024PyRIT: A Framework for Security Risk Identification and Red Teaming in Generative AI Systemcited by 31
  19. 2025Persuade Me if You Can: A Framework for Evaluating Persuasion Effectiveness and Susceptibility Among Large Language Modelscited by 27
  20. 2026ClawGuard: A Runtime Security Framework for Tool-Augmented LLM Agents Against Indirect Prompt Injectioncited by 27
  21. 2026A Survey on Long-Term Memory Security in LLM Agents: Attacks, Defenses, and Governance Across the Memory Lifecyclecited by 27
  22. 2024Adversarial Tuning: Defending Against Jailbreak Attacks for LLMscited by 26
  23. 2025The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMscited by 26
  24. 2024Faster-GCG: Efficient Discrete Optimization Jailbreak Attacks against Aligned Large Language Modelscited by 25
  25. 2025Visual Contextual Attack: Jailbreaking MLLMs with Image-Driven Context Injectioncited by 25
  26. 2025FORTRESS: Frontier Risk Evaluation for National Security and Public Safetycited by 24
  27. 2025Lessons From Red Teaming 100 Generative AI Productscited by 23
  28. 2025SafeDialBench: A Fine-Grained Safety Evaluation Benchmark for Large Language Models in Multi-Turn Dialogues with Diverse Jailbreak Attackscited by 23
  29. 2025Reasoning as an Adaptive Defense for Safetycited by 23
  30. 2025AgentBound: Securing Execution Boundaries of AI Agentscited by 23
  31. 2024Breaking ReAct Agents: Foot-in-the-Door Attack Will Get You Incited by 22
  32. 2025SoK: Evaluating Jailbreak Guardrails for Large Language Modelscited by 21
  33. 2026MUZZLE: Adaptive Agentic Red-Teaming of Web Agents Against Indirect Prompt Injection Attackscited by 21
  34. 2024Dataset and Lessons Learned from the 2024 SaTML LLM Capture-the-Flag Competitioncited by 20
  35. 2025X-Boundary: Establishing Exact Safety Boundary to Shield LLMs from Multi-Turn Jailbreaks without Compromising Usabilitycited by 20
  36. 2025Steering Dialogue Dynamics for Robustness against Multi-turn Jailbreaking Attackscited by 20
  37. 2025SafeKey: Amplifying Aha-Moment Insights for Safety Reasoningcited by 20
  38. 2026Your Agent, Their Asset: A Real-World Safety Analysis of OpenClawcited by 20

50 most cited of 390 All 390 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily
  2. 2023Tricking LLMs into Disobedience: Formalizing, Analyzing, and Detecting Jailbreaks
  3. 2024A Comprehensive Study of Jailbreak Attack versus Defense for Large Language Models
  4. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Survey
  5. 2024All in How You Ask for It: Simple Black-Box Method for Jailbreak Attacks
  6. 2025Effective and Efficient Jailbreaks of Black-Box LLMs with Cross-Behavior Attacks
  7. 2024Intention Analysis Makes LLMs A Good Jailbreak Defender
  8. 2023Analyzing the Inherent Response Tendency of LLMs: Real-World Instructions-Driven Jailbreak