mlsanityData to 7 Oct 2026

Immune: Improving Safety Against Jailbreaks in Multi-modal LLMs via Inference-Time Alignment

arXiv 2411.18688 ↗First posted 27 Nov 2024cs.CR cs.AI cs.LGcited by 42

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 3 more
Abstract (arXiv)

With the widespread deployment of Multimodal Large Language Models (MLLMs) for visual-reasoning tasks, improving their safety has become crucial. Recent research indicates that despite training-time safety alignment, these models remain vulnerable to jailbreak attacks. In this work, we first highlight an important safety gap to describe that alignment achieved solely through safety training may be insufficient against jailbreak attacks. To address this vulnerability, we propose Immune, an inference-time defense framework that leverages a safe reward model through controlled decoding to defend against jailbreak attacks. Additionally, we provide a mathematical characterization of Immune, offering insights on why it improves safety against jailbreaks. Extensive evaluations on diverse jailbreak benchmarks using recent MLLMs reveal that Immune effectively enhances model safety while preserving the model's original capabilities. For instance, against text-based jailbreak attacks on LLaVA-1.6, Immune reduces the attack success rate by 57.82% and 16.78% compared to the base MLLM and state-of-the-art defense strategy, respectively.

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 77 of its 92 references that are in the map. First posted on arXiv 27 Nov 2024.

Builds on

77 of 92 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs12 Jan
  2. 2023Cognitive Overload: Jailbreaking Large Language Models with Overloaded Logical Thinking16 Nov

Other references · 75

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  5. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  6. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  7. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  8. 2020Learning to summarize from human feedbackcited by 2,562
  9. 2015VQA: Visual Question Answeringcited by 2,489
  10. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  11. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,461
  12. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
38 more
  1. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
  2. 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
  3. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  4. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  5. 2022Red Teaming Language Models with Language Modelscited by 983
  6. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
  7. 2023A Comprehensive Overview of Large Language Modelscited by 639
  8. 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 505
  9. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 494
  10. 2024RewardBench: Evaluating Reward Models for Language Modelingcited by 480
  11. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 478
  12. 2024WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMscited by 460
  13. 2023RRHF: Rank Responses to Align Language Models with Human Feedback without tearscited by 454
  14. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 450
  15. 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deepcited by 450
  16. 2019On Evaluating Adversarial Robustnesscited by 442
  17. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generationcited by 439
  18. 2023RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedbackcited by 404
  19. 2024Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothingcited by 341
  20. 2017Adversarial Attacks on Neural Network Policiescited by 314
  21. 2023Visual Adversarial Examples Jailbreak Aligned Large Language Modelscited by 307
  22. 2023On Evaluating Adversarial Robustness of Large Vision-Language Modelscited by 281
  23. 2023MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Modelscited by 255
  24. 2024HelpSteer2: Open-source dataset for training top-performing reward modelscited by 232
  25. 2018A Comprehensive Survey of Deep Learning for Image Captioningcited by 213
  26. 2024JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attackscited by 208
  27. 2023Automatically Auditing Large Language Models via Discrete Optimizationcited by 187
  28. 2023How Robust is Google's Bard to Adversarial Image Attacks?cited by 177
  29. 2023Macaw-LLM: Multi-Modal Language Modeling with Image, Audio, Video, and Text Integrationcited by 168
  30. 2024COLD-Attack: Jailbreaking LLMs with Stealthiness and Controllabilitycited by 162
  31. 2024Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Modelscited by 160
  32. 2023RAIN: Your Language Models Can Align Themselves without Finetuningcited by 149
  33. 2023On the Adversarial Robustness of Multi-Modal Foundation Modelscited by 142
  34. 2023Controlled Decoding from Language Modelscited by 133
  35. 2023Silkie: Preference Distillation for Large Visual Language Modelscited by 128
  36. 2024Jailbreaking Attack against Multimodal Large Language Modelcited by 128
  37. 2016Visual Question Answering: A Survey of Methods and Datasetscited by 123
  38. 2024AdaShield: Safeguarding Multimodal Large Language Models from Structure-based Attack via Adaptive Shield Promptingcited by 113

50 most cited of 75 All 75 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 19; 2026: 22.

42 papers.

Sort:

By area:

  1. 2025A Survey of Safety on Large Vision-Language Models: Attacks, Defenses and Evaluationscited by 66
  2. 2025Safety in Large Reasoning Models: A Surveycited by 58
  3. 2025VPI-Bench: Visual Prompt Injection Attacks for Computer-Use Agentscited by 42
  4. 2025GuardReasoner-VL: Safeguarding VLMs via Reinforced Reasoningcited by 38
  5. 2025Automating Steering for Safe Multimodal Large Language Modelscited by 17
  6. 2025Omni-SafetyBench: A Benchmark for Safety Evaluation of Audio-Visual Large Language Modelscited by 13
  7. 2025SafePTR: Token-Level Jailbreak Defense in Multimodal LLMs via Prune-then-Restore Mechanismcited by 9
  8. 2025SaFeR-VLM: Toward Safety-aware Fine-grained Reasoning in Multimodal Modelscited by 8
  9. 2025Relic: Enhancing Reward Model Generalization for Low-Resource Indic Languages with Few-Shot Examplescited by 6
  10. 2025GrAInS: Gradient-based Attribution for Inference-Time Steering of LLMs and VLMscited by 6
  11. 2025Steering Multimodal Large Language Models Decoding for Context-Aware Safetycited by 6
  12. 2025EcoAlign: An Economically Rational Framework for Efficient LVLM Alignmentcited by 5
30 more
  1. 2026Evolving Contextual Safety in Multi-Modal Large Language Models via Inference-Time Self-Reflective Memorycited by 5
  2. 2025Attention Misses Visual Risk: Risk-Adaptive Steering for Multimodal Safety Alignmentcited by 4
  3. 2026MTMCS-Bench: Evaluating Contextual Safety of Multimodal Large Language Models in Multi-Turn Dialoguescited by 4
  4. 2026Visual Inception: Compromising Long-term Planning in Agentic Recommenders via Multimodal Memory Poisoningcited by 4
  5. 2024LIAR: Leveraging Inference Time Alignment (Best-of-N) to Jailbreak LLMs in Secondscited by 3
  6. 2025The Safety Reminder: A Soft Prompt to Reactivate Delayed Safety Awareness in Vision-Language Modelscited by 3
  7. 2025SafeR-CLIP: Mitigating NSFW Content in Vision-Language Models While Preserving Pre-Trained Knowledgecited by 3
  8. 2026WARD: Adversarially Robust Defense of Web Agents Against Prompt Injectionscited by 3
  9. 2025Self-Aware Safety Augmentation: Leveraging Internal Semantic Understanding to Enhance Safety in Vision-Language Modelscited by 2
  10. 2025Empowering Multimodal LLMs with External Tools: A Comprehensive Surveycited by 2
  11. 2026Risk Awareness Injection: Calibrating Vision-Language Models for Safety without Compromising Utilitycited by 2
  12. 2026Principled Steering via Null-space Projection for Jailbreak Defense in Vision-Language Modelscited by 2
  13. 2026CASA: Classification Augmented with Safety Attention for Robust Multimodal Alignmentcited by 2
  14. 2026TrajGuard: Streaming Hidden-state Trajectory Detection for Decoding-time Jailbreak Defensecited by 2
  15. 2026Safe responses matter: Output-aware safety guardrail mitigate over-refusal in MLLMscited by 2
  16. 2025Dynamic Token Reweighting for Robust Vision-Language Modelscited by 1
  17. 2025Robust Diagram Reasoning: A Framework for Enhancing LVLM Performance on Visually Perturbed Scientific Diagramscited by 1
  18. 2026Are GUI Agents Focused Enough? Automated Distraction via Semantic-level UI Element Injectioncited by 1
  19. 2026SafeCap: Improving LVLM Safety with Image Captioning Reinforcement Learningcited by 1
  20. 2026Feature-Space Smoothing: Certified Robustness of Deep Representationscited by 0
  21. 2026Two Birds, One Projection: Harmonizing Safety and Utility in LVLMs via Inference-time Feature Projectioncited by 0
  22. 2026New Wide-Net-Casting Jailbreak Attacks Risk Large Modelscited by 0
  23. 2026Prosody-driven Jailbreaks in Audio LLMs: A Controlled Study and Mechanistic Analysiscited by 0
  24. 2026SafeNexus: Discovering and Steering Modality-Universal Safety Neurons in MLLMscited by 0
  25. 2026Decoy Images Amplify Caption-Mediated Defenses Against Encoded Jailbreakscited by 0
  26. 2026Evolving Safety Landscape of Multi-modal Large Language Models: A Survey of Emerging Threats and Safeguardscited by 0
  27. 2026COMIC: Reference-Aware Safety Gating for Multimodal Large Language Modelscited by 0
  28. 2026ReFrame: Evidence-Guided Test-Time Safety Alignment in Multimodal Large Language Modelscited by 0
  29. 2026SafeRI: Recognition and Intervention for Token-Level Safety Intervention in Large Vision Language Modelscited by 0
  30. 2026DIVA: Exploiting Cross-Step Conditional Propagation for Visual Jailbreaks in Discrete Diffusion Vision-Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Test-Time Immunization: A Universal Defense Framework Against Jailbreaks for (Multimodal) Large Language Models
  2. 2025ARMOR: Aligning Secure and Safe Large Language Models via Meticulous Reasoning
  3. 2026Defending Large Language Models Against Jailbreak Attacks via In-Decoding Safety-Awareness Probing
  4. 2025PandaGuard: Systematic Evaluation of LLM Safety against Jailbreaking Attacks
  5. 2024Defending Large Language Models Against Jailbreak Attacks via Layer-specific Editing
  6. 2025Do Methods to Jailbreak and Defend LLMs Generalize Across Languages?
  7. 2026Causal Front-Door Adjustment for Robust Jailbreak Attacks on LLMs
  8. 2026Circuit Discovery Helps Detect LLM Jailbreaking: A Mechanistic Interpretability Study