mlsanityData to 7 Oct 2026

JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attacks

arXiv 2404.03027 ↗First posted 3 Apr 2024cs.CR cs.AI cs.CLcited by 208

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

also in 1 more
Abstract (arXiv)

With the rapid advancements in Multimodal Large Language Models (MLLMs), securing these models against malicious inputs while aligning them with human values has emerged as a critical challenge. In this paper, we investigate an important and unexplored question of whether techniques that successfully jailbreak Large Language Models (LLMs) can be equally effective in jailbreaking MLLMs. To explore this issue, we introduce JailBreakV-28K, a pioneering benchmark designed to assess the transferability of LLM jailbreak techniques to MLLMs, thereby evaluating the robustness of MLLMs against diverse jailbreak attacks. Utilizing a dataset of 2, 000 malicious queries that is also proposed in this paper, we generate 20, 000 text-based jailbreak prompts using advanced jailbreak attacks on LLMs, alongside 8, 000 image-based jailbreak inputs from recent MLLMs jailbreak attacks, our comprehensive dataset includes 28, 000 test cases across a spectrum of adversarial scenarios. Our evaluation of 10 open-source MLLMs reveals a notably high Attack Success Rate (ASR) for attacks transferred from LLMs, highlighting a critical vulnerability in MLLMs that stems from their text-processing capabilities. Our findings underscore the urgent need for future research to address alignment vulnerabilities in MLLMs from both textual and visual inputs.

Key facts. Cited by 208 papers in the map (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 3 of its 57 references that are in the map. First posted on arXiv 3 Apr 2024.

Builds on

3 of 57 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023How Robust is Google's Bard to Adversarial Image Attacks?21 Sep

Other references · 2

  1. 2023Qwen Technical Reportcited by 3,273
  2. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 39; 2025: 103; 2026: 66.

208 papers.

Sort:

By area:

  1. 2024AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMscited by 185
  2. 2025NVIDIA Nemotron Nano 2: An Accurate and Efficient Hybrid Mamba-Transformer Reasoning Modelcited by 94
  3. 2025Nemotron 3 Nano: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoningcited by 85
  4. 2024SPA-VL: A Comprehensive Safety Preference Alignment Dataset for Vision Language Modelcited by 82
  5. 2024A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trendscited by 82
  6. 2025AGrail: A Lifelong Agent Guardrail with Effective and Adaptive Safety Detectioncited by 82
  7. 2024A Survey on Benchmarks of Multimodal Large Language Modelscited by 81
  8. 2025STAIR: Improving Safety Alignment with Introspective Reasoningcited by 72
  9. 2024JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Modelscited by 67
  10. 2024T2VSafetyBench: Evaluating the Safety of Text-to-Video Generative Modelscited by 66
  11. 2025A Survey of Safety on Large Vision-Language Models: Attacks, Defenses and Evaluationscited by 66
  12. 2024Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agentscited by 63
38 more
  1. 2024Visual-RolePlay: Universal Jailbreak Attack on MultiModal Large Language Models via Role-playing Image Charactercited by 61
  2. 2024VLSBench: Unveiling Visual Leakage in Multimodal Safetycited by 59
  3. 2025RealSafe-R1: Safety-Aligned DeepSeek-R1 without Compromising Reasoning Capabilitycited by 58
  4. 2025ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inferencecited by 57
  5. 2024Jailbreak Large Vision-Language Models Through Multi-Modal Linkagecited by 53
  6. 2024MultiTrust: A Comprehensive Benchmark Towards Trustworthy Multimodal Large Language Modelscited by 51
  7. 2024Bag of Tricks: Benchmarking of Jailbreak Attacks on LLMscited by 47
  8. 2024A Survey on Evaluation of Multimodal Large Language Modelscited by 46
  9. 2025A Survey of State of the Art Large Vision Language Models: Alignment, Benchmark, Evaluations and Challengescited by 46
  10. 2025AlphaSteer: Learning Refusal Steering with Principled Null-Space Constraintcited by 45
  11. 2024Immune: Improving Safety Against Jailbreaks in Multi-modal LLMs via Inference-Time Alignmentcited by 42
  12. 2024CLEAR: Character Unlearning in Textual and Visual Modalitiescited by 36
  13. 2024Unveiling the Safety of GPT-4o: An Empirical Study using Jailbreak Attackscited by 35
  14. 2024IDEATOR: Jailbreaking and Benchmarking Large Vision-Language Models Using Themselvescited by 34
  15. 2024Jailbreaking and Mitigation of Vulnerabilities in Large Language Modelscited by 33
  16. 2026Nemotron 3 Super: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoningcited by 33
  17. 2025Towards Robust and Secure Embodied AI: A Survey on Vulnerabilities and Attackscited by 32
  18. 2024Jailbreak Attacks and Defenses against Multimodal Generative Models: A Surveycited by 31
  19. 2025SafeEraser: Enhancing Safety in Multimodal Large Language Models through Multimodal Machine Unlearningcited by 31
  20. 2024Heuristic-Induced Multimodal Risk Distribution Jailbreak Attack for Multimodal Large Language Modelscited by 29
  21. 2025HiddenDetect: Detecting Jailbreak Attacks against Large Vision-Language Models via Monitoring Hidden Statescited by 29
  22. 2024JailbreakEval: An Integrated Toolkit for Evaluating Jailbreak Attempts Against Large Language Modelscited by 28
  23. 2024BlueSuffix: Reinforced Blue Teaming for Vision-Language Models Against Jailbreak Attackscited by 28
  24. 2024A Survey on Multimodal Benchmarks: In the Era of Large AI Modelscited by 27
  25. 2025Video-SafetyBench: A Benchmark for Safety Evaluation of Video LVLMscited by 25
  26. 2024Safe Inputs but Unsafe Output: Benchmarking Cross-modality Safety Alignment of Large Vision-Language Modelcited by 24
  27. 2024Cross-modality Information Check for Detecting Jailbreaking in Multimodal Large Language Modelscited by 24
  28. 2025Towards Understanding the Safety Boundaries of DeepSeek Models: Evaluation and Findingscited by 24
  29. 2025The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMscited by 24
  30. 2024Cross-Modal Safety Alignment: Is textual unlearning all you need?cited by 22
  31. 2024From LLMs to MLLMs: Exploring the Landscape of Multimodal Jailbreakingcited by 22
  32. 2024Adversarial Attacks of Vision Tasks in the Past 10 Years: A Surveycited by 22
  33. 2025Rethinking Bottlenecks in Safety Fine-Tuning of Vision Language Modelscited by 21
  34. 2025Frontier AI's Impact on the Cybersecurity Landscapecited by 21
  35. 2025EmoAgent: Assessing and Safeguarding Human-AI Interaction for Mental Health Safetycited by 21
  36. 2024The VLLM Safety Paradox: Dual Ease in Jailbreak Attack and Defensecited by 20
  37. 2025Inverse Reinforcement Learning with Dynamic Reward Scaling for LLM Alignmentcited by 20
  38. 2025Think in Safety: Unveiling and Mitigating Safety Alignment Collapse in Multimodal Large Reasoning Modelcited by 18

50 most cited of 208 All 208 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024textitMMJ-Bench: A Comprehensive Study on Jailbreak Attacks and Defenses for Multimodal Large Language Models
  2. 2025Probabilistic Modeling of Jailbreak on Multimodal LLMs: From Quantification to Application
  3. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Survey
  4. 2025Align is not Enough: Multimodal Universal Jailbreak Attack against Multimodal Large Language Models
  5. 2024Efficient LLM-Jailbreaking via Multimodal-LLM Jailbreak
  6. 2024Divide and Conquer: A Hybrid Strategy Defeats Multimodal Large Language Models
  7. 2025The Tower of Babel Revisited: Multilingual Jailbreak Prompts on Closed-Source Large Language Models
  8. 2024Jailbreaking Attack against Multimodal Large Language Model