mlsanityData to 14 Oct 2026

Diversity Helps Jailbreak Large Language Models

arXiv 2411.04223 ↗First posted 6 Nov 2024cs.CLcited by 10

On the map: LLM safety and security → Jailbreaking

Abstract (arXiv)

We have uncovered a powerful jailbreak technique that leverages large language models' ability to diverge from prior context, enabling them to bypass safety constraints and generate harmful outputs. By simply instructing the LLM to deviate and obfuscate previous attacks, our method dramatically outperforms existing approaches, achieving up to a 62.83% higher success rate in compromising ten leading chatbots, including GPT-4, Gemini, and Llama, while using only 12.9% of the queries. This revelation exposes a critical flaw in current LLM safety training, suggesting that existing methods may merely mask vulnerabilities rather than eliminate them. Our findings sound an urgent alarm for the need to revolutionize testing methodologies to ensure robust and reliable LLM security.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 44 of its 54 references that are in the map. First posted on arXiv 6 Nov 2024.

Builds on

44 of 54 references.

Closest ancestors · 16

Same direction, newest first

  1. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Survey5 Jul
  2. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks2 Apr
  3. 2024AutoDefense: Multi-Agent LLM Defense against Jailbreak Attacks2 Mar
  4. 2024Making Them Ask and Answer: Jailbreaking Large Language Models in Few Queries via Disguise and Reconstruction28 Feb
  5. 2024Leveraging the Context through Multi-Round Interactions for Jailbreaking Attacks14 Feb
  6. 2024Weak-to-Strong Jailbreaking on Large Language Models30 Jan
  7. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs12 Jan
  8. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automatically4 Dec
  9. 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
  10. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generation10 Oct
  11. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks5 Oct
  12. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts19 Sep
4 more
  1. 2023Open Sesame! Universal Black Box Jailbreaking of Large Language Models4 Sep
  2. 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
  3. 2023MasterKey: Automated Jailbreak Across Multiple Large Language Model Chatbots16 Jul
  4. 2023Jailbroken: How Does LLM Safety Training Fail?5 Jul

Other references · 28

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  6. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  7. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  8. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  9. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  10. 2023Mistral 7Bcited by 3,164
  11. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  12. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
16 more
  1. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
  2. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,392
  3. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 1,006
  4. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 860
  5. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 729
  6. 2022Improving alignment of dialogue agents via targeted human judgementscited by 531
  7. 2023Multi-step Jailbreaking Privacy Attacks on ChatGPTcited by 333
  8. 2023Are aligned neural networks adversarially aligned?cited by 315
  9. 2023Large Language Models are Zero-Shot Rankers for Recommender Systemscited by 309
  10. 2023Pretraining Language Models with Human Preferencescited by 247
  11. 2023Automatically Auditing Large Language Models via Discrete Optimizationcited by 191
  12. 2019Build it Break it Fix it for Dialogue Safety: Robustness from Adversarial Human Attackcited by 176
  13. 2023AART: AI-Assisted Red-Teaming with Diverse Data Generation for New LLM-powered Applicationscited by 54
  14. 2023Hijacking Large Language Models via Adversarial In-Context Learningcited by 45
  15. 2024SPML: A DSL for Defending Language Models Against Prompt Attackscited by 34
  16. 2024ROSE Doesn't Do That: Boosting the Safety of Instruction-Tuned Large Language Models with Reverse Prompt Contrastive Decodingcited by 32

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 6.

10 papers.

  1. 2025LARGO: Latent Adversarial Reflection through Gradient Optimization for Jailbreaking LLMscited by 7
  2. 2025Proactive defense against LLM Jailbreakcited by 7
  3. 2026Exposing Long-Tail Safety Failures in Large Language Models through Efficient Diverse Response Samplingcited by 2
  4. 2025Self-Instruct Few-Shot Jailbreaking: Decompose the Attack into Pattern and Behavior Learningcited by 1
  5. 2026Break Me If You Can: Self-Jailbreaking of Aligned LLMs via Lexical Insertion Promptingcited by 1
  6. 2025Jailbreak-Zero: A Path to Pareto Optimal Red Teaming for Large Language Modelscited by 1
  7. 2026Prompt Injection Detection is Regime-Dependent: A Deployment-Aware Evaluation with Interpretable Structural Signalscited by 0
  8. 2026OTTER: A Red-Teaming System for Toxicity-Evading Jailbreak Prompt Optimizationcited by 0
  9. 2026The Safeguard Worked. Is the LLM System Safer?cited by 0
  10. 2026REPRO: Reflective Evolutionary Prompt Optimizer for Automated LLM Red-Teamingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024A Comprehensive Study of Jailbreak Attack versus Defense for Large Language Models
  2. 2024WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Models
  3. 2024Jailbreaking and Mitigation of Vulnerabilities in Large Language Models
  4. 2024SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decoding
  5. 2026Defending Large Language Models Against Jailbreak Attacks via In-Decoding Safety-Awareness Probing
  6. 2025Dark LLMs: The Growing Threat of Unaligned AI Models
  7. 2025Layer-Level Self-Exposure and Patch: Affirmative Token Mitigation for Jailbreak Attack Defense
  8. 2025You Can't Eat Your Cake and Have It Too: The Performance Degradation of LLMs with Jailbreak Defense