mlsanityData to 7 Oct 2026

Jailbreaking

911 papers5 this weekHigh confidence in this grouping: the papers form one clear topicHigh confidence in this grouping: the papers form one clear topic · About

Keywords: jailbreak, llms, attacks, attack, language

Key facts. Jailbreaking: 911 papers on arXiv since 2023 (data through 7 Oct 2026). 620 of its 911 mapped papers (68%) cite “Universal and Transferable Adversarial Attacks on Aligned Language Models” (2023, arXiv 2307.15043). 540 of its 911 mapped papers (59%) cite “Jailbreaking Black Box Large Language Models in Twenty Queries” (2023, arXiv 2310.08419). 21 new papers 24 Sep–7 Oct.

What’s new

24 Sep – 7 Oct 2026

21 new papers, 24 Sep – 7 Oct; 5 this week.

  1. This week · 5
  2. PatchBench: Measuring Collateral Damage in Activation Patching
  3. From Expected Harmfulness to Likelihood: A Probabilistic Reformulation of Jailbreaking LLM Agents
  4. Secure Speculative Decoding for Large Language Models
  5. Jailbreaking Open-Weight LLMs via Random Embedding Perturbations
  6. Reward Stealing Attack on Large Language Models also fits: Safety alignment, refusal and guardrails
  7. Earlier · 16
  8. Which Image Property Carries the Jailbreak? A Controlled Dissection of Image-to-Text Jailbreaks also fits: Vision-language and text-to-image safety
  9. Red-TTT: Test-Time Training for Automated Jailbreaking Large Language Models
  10. Reactivating Alignment: Defending LLMs from Jailbreaks via Intention-Aware Input-Output Matching
  11. Target-free Latent Safety Alignment also fits: Safety alignment, refusal and guardrails
  12. MLCommons Jailbreak Benchmark v1.0
All 21
  1. Intent-Hiding Jailbreaks: An Information-Theoretic Framework for Compositional Attacks
  2. MOMAT: Mixture of Multiple Atlases for Low-Power Jailbreak Defense of Quantized LLMs also fits: Safety alignment, refusal and guardrails
  3. CodeMimicry: Exploiting Safety Generalization Lag in Large Language Models via Structured Code Completion
  4. SceneJail: Exploiting Video Scenario Context to Jailbreak Multimodal LLMs also fits: Vision-language and text-to-image safety
  5. Controlled Decoding Attacks on Black-Box LLMs
  6. Does the Unsafe Gradient Survive a Conversation? On the Fragility of Gradient-Based Jailbreak Detection in Multi-Turn Dialogue
  7. Escaping Alignment: A Physical Trap Model of Best-of-N Jailbreaking
  8. Monitor Jailbreaking: Evading Chain-of-Thought Monitoring Without Encoded Reasoning also fits: Safety alignment, refusal and guardrails
  9. TempQ-Jail: Query-Constrained Candidate Ranking for Text-to-Video Jailbreak Attacks
  10. Why Jailbreaks Succeed in Diffusion Language Models: An Energy Landscape Analysis
  11. Prefilling the Reasoning Channel: Output-Prefix Attacks on Reasoning LLMs also fits: Prompt injection and agent security

Foundations

Most cited by papers here

  1. Universal and Transferable Adversarial Attacks on Aligned Language Models2023cited by 620 here
  2. Jailbreaking Black Box Large Language Models in Twenty Queries2023cited by 540 here
  3. Jailbroken: How Does LLM Safety Training Fail?2023cited by 441 here
  4. AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models2023cited by 296 here
  5. Tree of Attacks: Jailbreaking Black-Box LLMs Automatically2023cited by 290 here

Lineage

The papers this direction builds on, oldest first.

Each paper on the line cites the one before.

  1. 2022Red Teaming Language Models with Language Modelscited by 133 here
  2. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 441 here
  3. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 620 here
  4. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Ciphercited by 231 here
  5. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 296 here
  6. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 211 here
  7. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 540 here
  8. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 290 here
  9. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 356 here
  10. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 184 here
  11. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 181 here
13 more it builds on, not in this chain
  1. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 175 here
  2. 2023"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Modelscited by 157 here
  3. 2023Detecting Language Model Attacks with Perplexitycited by 167 here
  4. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 247 here
  5. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 199 here
  6. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrationscited by 196 here
  7. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generationcited by 165 here
  8. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreakercited by 193 here
  9. 2023A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easilycited by 155 here
  10. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 236 here
  11. 2024SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decodingcited by 114 here
  12. 2024ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMscited by 119 here
  13. 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 130 here

The 24 most cited of 1,197.

Browse

30 of 911, ranked 6–35

  1. 6GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts2023247
  2. 7How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs2024236
  3. 8GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher2023231
  4. 9SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks2023211
  5. 10Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations2023196
  6. 11DeepInception: Hypnotize Large Language Model to Be Jailbreaker2023193
  7. 12JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models2024184
  8. 13Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks2024181
  9. 14Jailbreaking ChatGPT via Prompt Engineering: An Empirical Study2023175
  10. 15Detecting Language Model Attacks with Perplexity2023167
All 30
  1. 16Catastrophic Jailbreak of Open-source LLMs via Exploiting Generation2023165
  2. 17"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models2023157
  3. 18A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily2023155
  4. 19Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attack2024130
  5. 20SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decoding2024114
  6. 21A StrongREJECT for Empty Jailbreaks2024108
  7. 22Low-Resource Languages Jailbreak GPT-42023108
  8. 23AmpleGCG: Learning a Universal and Transferable Generative Model of Adversarial Suffixes for Jailbreaking Both Open and Closed LLMs2024105
  9. 24Defending Against Alignment-Breaking Attacks via Robustly Aligned LLM2023105
  10. 25Multilingual Jailbreak Challenges in Large Language Models202397
  11. 26Certifying LLM Safety against Adversarial Prompting202394
  12. 27AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMs202491
  13. 28Jailbreak Attacks and Defenses Against Large Language Models: A Survey202490
  14. 29MasterKey: Automated Jailbreak Across Multiple Large Language Model Chatbots202390
  15. 30Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritization202390
  16. 31Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attacks202479
  17. 32COLD-Attack: Jailbreaking LLMs with Stealthiness and Controllability202478
  18. 33Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulation202377
  19. 34Open Sesame! Universal Black Box Jailbreaking of Large Language Models202376
  20. 35AdvPrompter: Fast Adaptive Adversarial Prompting for LLMs202469