mlsanityData to 7 Oct 2026

JailbreakRadar: Comprehensive Assessment of Jailbreak Attacks Against LLMs

arXiv 2402.05668 ↗First posted 8 Feb 2024cs.CR cs.AI cs.CLcited by 87

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

Jailbreak attacks aim to bypass the LLMs' safeguards. While researchers have proposed different jailbreak attacks in depth, they have done so in isolation -- either with unaligned settings or comparing a limited range of methods. To fill this gap, we present a large-scale evaluation of various jailbreak attacks. We collect 17 representative jailbreak attacks, summarize their features, and establish a novel jailbreak attack taxonomy. Then we conduct comprehensive measurement and ablation studies across nine aligned LLMs on 160 forbidden questions from 16 violation categories. Also, we test jailbreak attacks under eight advanced defenses. Based on our taxonomy and experiments, we identify some important patterns, such as heuristic-based attacks could achieve high attack success rates but are easy to mitigate by defenses, causing low practicality. Our study offers valuable insights for future research on jailbreak attacks and defenses. We hope our work could help the community avoid incremental work and serve as an effective benchmark tool for practitioners.

Key facts. Cited by 87 papers in the map (data through 7 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 46 of its 73 references that are in the map. First posted on arXiv 8 Feb 2024.

Builds on

46 of 73 references.

Closest ancestors · 13

Same direction, newest first

  1. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automatically4 Dec
  2. 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
  3. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generation10 Oct
  4. 2023Multilingual Jailbreak Challenges in Large Language Models10 Oct
  5. 2023Low-Resource Languages Jailbreak GPT-43 Oct
  6. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts19 Sep
  7. 2023Certifying LLM Safety against Adversarial Prompting6 Sep
  8. 2023Detecting Language Model Attacks with Perplexity27 Aug
  9. 2023Robustness Over Time: Understanding Adversarial Examples' Effectiveness on Longitudinal Versions of Large Language Models15 Aug
  10. 2023"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models7 Aug
  11. 2023Jailbroken: How Does LLM Safety Training Fail?5 Jul
  12. 2023Tricking LLMs into Disobedience: Formalizing, Analyzing, and Detecting Jailbreaks24 May
1 more
  1. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Study23 May

Other references · 33

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  7. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  8. 2020Extracting Training Data from Large Language Modelscited by 2,144
  9. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  10. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  11. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  12. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
21 more
  1. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  2. 2022Red Teaming Language Models with Language Modelscited by 983
  3. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
  4. 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 663
  5. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
  6. 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 505
  7. 2022A Holistic Approach to Undesired Content Detection in the Real Worldcited by 337
  8. 2023Multi-step Jailbreaking Privacy Attacks on ChatGPTcited by 320
  9. 2023Analyzing Leakage of Personally Identifiable Information in Language Modelscited by 278
  10. 2023Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attackscited by 268
  11. 2019Detecting AI Trojans Using Meta Neural Analysiscited by 192
  12. 2020BadNL: Backdoor Attacks against NLP Models with Semantic-preserving Improvementscited by 191
  13. 2023Unsafe Diffusion: On the Generation of Unsafe Images and Hateful Memes From Text-To-Image Modelscited by 171
  14. 2022Quantifying Privacy Risks of Masked Language Models Using Membership Inference Attackscited by 168
  15. 2022Truth Serum: Poisoning Machine Learning Models to Reveal Their Secretscited by 94
  16. 2023A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validationcited by 91
  17. 2023MGTBench: Benchmarking Machine-Generated Text Detectioncited by 90
  18. 2021Bad Characters: Imperceptible NLP Attackscited by 69
  19. 2021Spinning Language Models: Risks of Propaganda-As-A-Service and Countermeasurescited by 69
  20. 2023In ChatGPT We Trust? Measuring and Characterizing the Reliability of ChatGPTcited by 35
  21. 2024Reconstruct Your Previous Conversations! Comprehensively Investigating Privacy Leakage Risks in Conversations with GPT Modelscited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 33; 2025: 22; 2026: 32.

87 papers.

Sort:

By area:

  1. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Surveycited by 215
  2. 2024Improved Techniques for Optimization-Based Jailbreaking on Large Language Modelscited by 98
  3. 2024When LLMs Meet Cybersecurity: A Systematic Literature Reviewcited by 70
  4. 2024Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agentscited by 63
  5. 2024Get my drift? Catching LLM Task Drift with Activation Deltascited by 61
  6. 2024SelfDefend: LLMs Can Defend Themselves against Jailbreaking in a Practical Mannercited by 57
  7. 2024Bag of Tricks: Benchmarking of Jailbreak Attacks on LLMscited by 47
  8. 2024Voice Jailbreak Attacks Against GPT-4ocited by 39
  9. 2024Model Editing as a Robust and Denoised variant of DPO: A Case Study on Toxicitycited by 33
  10. 2024SHIELD: Evaluation and Defense Strategies for Copyright Compliance in LLM Text Generationcited by 33
  11. 2024JailbreakEval: An Integrated Toolkit for Evaluating Jailbreak Attempts Against Large Language Modelscited by 28
  12. 2025The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMscited by 24
38 more
  1. 2024Adversarial Tuning: Defending Against Jailbreak Attacks for LLMscited by 23
  2. 2024Shaping the Safety Boundaries: Understanding and Defending Against Jailbreaks in Large Language Modelscited by 23
  3. 2026SkillTrojan: Backdoor Attacks on Skill-Based Agent Systemscited by 21
  4. 2024PAPILLON: Efficient and Stealthy Fuzz Testing-Powered Jailbreaks for LLMscited by 20
  5. 2024Extracting Unlearned Information from LLMs with Activation Steeringcited by 19
  6. 2025T2VShield: Model-Agnostic Jailbreak Defense for Text-to-Video Modelscited by 17
  7. 2025SoK: Evaluating Jailbreak Guardrails for Large Language Modelscited by 17
  8. 2024Poisoned LangChain: Jailbreak LLMs by LangChaincited by 14
  9. 2024Legilimens: Practical and Unified Content Moderation for Large Language Model Servicescited by 13
  10. 2025JADES: A Universal Framework for Jailbreak Assessment via Decompositional Scoringcited by 12
  11. 2024Cannot or Should Not? Automatic Analysis of Refusal Composition in IFT/RLHF Datasets and Refusal Behavior of Black-Box LLMscited by 11
  12. 2024LLMs Can Defend Themselves Against Jailbreaking in a Practical Manner: A Vision Papercited by 9
  13. 2024PathSeeker: Exploring LLM Security Vulnerabilities with a Reinforcement Learning-Based Jailbreak Approachcited by 9
  14. 2024Insights and Current Gaps in Open-Source LLM Vulnerability Scanners: A Comparative Analysiscited by 9
  15. 2026HarmfulSkillBench: How Do Harmful Skills Weaponize Your Agents?cited by 9
  16. 2025LLM Security: Vulnerabilities, Attacks, Defenses, and Countermeasurescited by 8
  17. 2025EVA: Evolving Semantic Adversaries for Red-Teaming GUI Agents Against Environmental Injection Attackscited by 8
  18. 2024Jailbreaking LLMs with Arabic Transliteration and Arabizicited by 7
  19. 2025Bridging the Safety Gap: A Guardrail Pipeline for Trustworthy LLM Inferencescited by 7
  20. 2026The Promptware Kill Chain: How Prompt Injections Gradually Evolved Into a Multistep Malware Delivery Mechanismcited by 7
  21. 2025From ASR to ASP: Evaluating Prompt Attack Vulnerabilities Against Open-Source LLMscited by 6
  22. 2024BlackDAN: A Black-Box Multi-Objective Approach for Effective and Contextual Jailbreaking of Large Language Modelscited by 5
  23. 2026ICON: Intent-Context Coupling for Efficient Multi-Turn Jailbreak Attackcited by 5
  24. 2024TorchOpera: A Compound AI System for LLM Safetycited by 4
  25. 2025PandaGuard: Systematic Evaluation of LLM Safety against Jailbreaking Attackscited by 4
  26. 2025AntiDote: Bi-level Adversarial Training for Tamper-Resistant LLMscited by 4
  27. 2025Guarding the Guardrails: A Taxonomy-Driven Approach to Jailbreak Detectioncited by 4
  28. 2025SoK: Systematizing LLM Prompt Security: Taxonomies, Datasets, and Unified Evaluation of Attacks and Defensescited by 4
  29. 2026Hijacking Agent Memory: Stealthy Trojan Attacks Through Conversational Interactioncited by 4
  30. 2024Towards Universal and Black-Box Query-Response Only Attack on LLMs with QROAcited by 3
  31. 2024A Survey on Privacy Attacks Against Digital Twin Systems in AI-Roboticscited by 3
  32. 2024Logicbreaks: A Framework for Understanding Subversion of Rule-based Inferencecited by 3
  33. 2025Targeting Alignment: Extracting Safety Classifiers of Aligned LLMscited by 3
  34. 2025STShield: Single-Token Sentinel for Real-Time Jailbreak Detection in Large Language Modelscited by 3
  35. 2025Beyond Jailbreaks: Revealing Stealthier and Broader LLM Security Risks Stemming from Alignment Failurescited by 3
  36. 2026RASA: Routing-Aware Safety Alignment for Mixture-of-Experts Modelscited by 3
  37. 2026Benchmark of Benchmarks: Unpacking Influence and Code Repository Quality in LLM Safety Benchmarkscited by 3
  38. 2026Understanding LLM Behavior When Encountering User-Supplied Harmful Content in Harmless Taskscited by 3

50 most cited of 87 All 87 (JSON)