How Jailbreak Defenses Work and Ensemble? A Mechanistic Investigation
On the map: LLM safety and security → Jailbreaking
Abstract (arXiv)
Jailbreak attacks, where harmful prompts bypass generative models' built-in safety, raise serious concerns about model vulnerability. While many defense methods have been proposed, the trade-offs between safety and helpfulness, and their application to Large Vision-Language Models (LVLMs), are not well understood. This paper systematically examines jailbreak defenses by reframing the standard generation task as a binary classification problem to assess model refusal tendencies for both harmful and benign queries. We identify two key defense mechanisms: safety shift, which increases refusal rates across all queries, and harmfulness discrimination, which improves the model's ability to distinguish between harmful and benign inputs. Using these mechanisms, we develop two ensemble defense strategies-inter-mechanism ensembles and intra-mechanism ensembles-to balance safety and helpfulness. Experiments on the MM-SafetyBench and MOSSBench datasets with LLaVA-1.5 models show that these strategies effectively improve model safety or optimize the trade-off between safety and helpfulness.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Jailbreaking, one of 12 directions in LLM safety and security. Builds on 40 of its 59 references that are in the map. First posted on arXiv 20 Feb 2025.
Builds on
40 of 59 references.
Closest ancestors · 17
Same direction, newest first
- 2024JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attacks3 Apr
- 2024Defending Large Language Models against Jailbreak Attacks via Semantic Smoothing25 Feb
- 2024SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decoding14 Feb
- 2024Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fast13 Feb
- 2024Intention Analysis Makes LLMs A Good Jailbreak Defender12 Jan
- 2023JailGuard: A Universal Detection Framework for LLM Prompt-based Attacks17 Dec
- 2023Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritization15 Nov
- 2023Jailbreaking Black Box Large Language Models in Twenty Queries12 Oct
- 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generation10 Oct
- 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations10 Oct
- 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks5 Oct
- 2023Defending Against Alignment-Breaking Attacks via Robustly Aligned LLM18 Sep
5 more
- 2023Certifying LLM Safety against Adversarial Prompting6 Sep
- 2023Detecting Language Model Attacks with Perplexity27 Aug
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
- 2023Jailbroken: How Does LLM Safety Training Fail?5 Jul
- 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Study23 May
Other references · 23
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
- 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
- 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
- 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,239
- 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,154
- 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 712
- 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Promptscited by 398
- 2023Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructionscited by 387
- 2023Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Modelscited by 273
- 2023LLM Self Defense: By Self Examination, LLMs Know They Are Being Trickedcited by 244
11 more
- 2023How Robust is Google's Bard to Adversarial Image Attacks?cited by 183
- 2023Image Hijacks: Adversarial Images can Control Generative Models at Runtimecited by 172
- 2023RAIN: Your Language Models Can Align Themselves without Finetuningcited by 157
- 2023From ChatGPT to ThreatGPT: Impact of Generative AI in Cybersecurity and Privacycited by 143
- 2024AdaShield: Safeguarding Multimodal Large Language Models from Structure-based Attack via Adaptive Shield Promptingcited by 125
- 2024Eyes Closed, Safety On: Protecting Multimodal LLMs via Image-to-Text Transformationcited by 124
- 2024MLLM-Protector: Ensuring MLLM's Safety without Hurting Performancecited by 117
- 2023Dr.ICL: Demonstration-Retrieved In-context Learningcited by 72
- 2024MOSSBench: Is Your Multimodal Language Model Oversensitive to Safe Queries?cited by 30
- 2023ToViLaG: Your Visual-Language Generative Model is Also An Evildoercited by 27
- 2024From LLMs to MLLMs: Exploring the Landscape of Multimodal Jailbreakingcited by 24
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025Defending Large Language Models Against Jailbreak Exploits with Responsible AI Considerations
- 2025Defense-to-Attack: Bypassing Weak Defenses Enables Stronger Jailbreaks in Vision-Language Models
- 2025SAID: Safety-Aware Intent Defense via Prefix Probing for Large Language Models
- 2024Understanding Jailbreak Success: A Study of Latent Space Dynamics in Large Language Models
- 2025Behind the Mask: Benchmarking Camouflaged Jailbreaks in Large Language Models
- 2025SafeInt: Shielding Large Language Models from Jailbreak Attacks via Safety-Aware Representation Intervention
- 2025Scalable Defense against In-the-wild Jailbreaking Attacks with Safety Context Retrieval
- 2026Minimal, Local, Causal Explanations for Jailbreak Success in Large Language Models