mlsanityData to 7 Oct 2026

Jailbreaking Attack against Multimodal Large Language Model

arXiv 2402.02309 ↗First posted 4 Feb 2024cs.LG cs.CL cs.CRcited by 128

On the map: LLM safety and security → Vision-language and text-to-image safety

Abstract (arXiv)

This paper focuses on jailbreaking attacks against multi-modal large language models (MLLMs), seeking to elicit MLLMs to generate objectionable responses to harmful user queries. A maximum likelihood-based algorithm is proposed to find an image Jailbreaking Prompt (imgJP), enabling jailbreaks against MLLMs across multiple unseen prompts and images (i.e., data-universal property). Our approach exhibits strong model-transferability, as the generated imgJP can be transferred to jailbreak various models, including MiniGPT-v2, LLaVA, InstructBLIP, and mPLUG-Owl2, in a black-box manner. Moreover, we reveal a connection between MLLM-jailbreaks and LLM-jailbreaks. As a result, we introduce a construction-based method to harness our approach for LLM-jailbreaks, demonstrating greater efficiency than current state-of-the-art methods. The code is available here. Warning: some content generated by language models may be offensive to some readers.

Key facts. Cited by 128 papers in the map (data through 7 Oct 2026). Placed in Vision-language and text-to-image safety, one of 12 directions in LLM safety and security. Builds on 32 of its 45 references that are in the map. First posted on arXiv 4 Feb 2024.

Builds on

32 of 45 references.

References · 32

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2023Visual Instruction Tuningcited by 7,792
  5. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  6. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  7. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  8. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  9. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  10. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  11. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  12. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
20 more
  1. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  2. 2020Extracting Training Data from Large Language Modelscited by 2,144
  3. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  4. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  5. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  6. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  7. 2022Red Teaming Language Models with Language Modelscited by 983
  8. 2023MiniGPT-v2: large language model as a unified interface for vision-language multi-task learningcited by 617
  9. 2023mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaborationcited by 545
  10. 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 540
  11. 2021Persistent Anti-Muslim Bias in Large Language Modelscited by 376
  12. 2023Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discoverycited by 329
  13. 2023Are aligned neural networks adversarially aligned?cited by 305
  14. 2023Pretraining Language Models with Human Preferencescited by 245
  15. 2021Gradient-based Adversarial Attacks against Text Transformerscited by 233
  16. 2021On the Robustness of Vision Transformers to Adversarial Examplescited by 167
  17. 2023Image Hijacks: Adversarial Images can Control Generative Models at Runtimecited by 165
  18. 2021Towards Transferable Adversarial Attacks on Vision Transformerscited by 80
  19. 2021On Improving Adversarial Transferability of Vision Transformerscited by 70
  20. 2022Patch-Fool: Are Vision Transformers Always Robust Against Adversarial Perturbations?cited by 47

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 44; 2025: 59; 2026: 25.

128 papers.

Sort:

By area:

  1. 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 467
  2. 2024Latent Adversarial Training Improves Robustness to Persistent Harmful Behaviors in LLMscited by 158
  3. 2024Images are Achilles' Heel of Alignment: Exploiting Visual Vulnerabilities for Jailbreaking Multimodal Large Language Modelscited by 121
  4. 2024AdaShield: Safeguarding Multimodal Large Language Models from Structure-based Attack via Adaptive Shield Promptingcited by 113
  5. 2024Jailbreak Vision Language Models via Bi-Modal Adversarial Promptcited by 103
  6. 2024SPA-VL: A Comprehensive Safety Preference Alignment Dataset for Vision Language Modelcited by 82
  7. 2024A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trendscited by 82
  8. 2024JailbreakZoo: Survey, Landscapes, and Horizons in Jailbreaking Large Language and Vision-Language Modelscited by 67
  9. 2025A Survey of Safety on Large Vision-Language Models: Attacks, Defenses and Evaluationscited by 66
  10. 2024Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agentscited by 63
  11. 2025Safety at Scale: A Comprehensive Survey of Large Model and Agent Safetycited by 63
  12. 2024Visual-RolePlay: Universal Jailbreak Attack on MultiModal Large Language Models via Role-playing Image Charactercited by 61
38 more
  1. 2024SelfDefend: LLMs Can Defend Themselves against Jailbreaking in a Practical Mannercited by 57
  2. 2024Don't Say No: Jailbreaking LLM by Suppressing Refusalcited by 53
  3. 2024MultiTrust: A Comprehensive Benchmark Towards Trustworthy Multimodal Large Language Modelscited by 51
  4. 2025A Survey of State of the Art Large Vision Language Models: Alignment, Benchmark, Evaluations and Challengescited by 46
  5. 2024Steering Away from Harm: An Adaptive Approach to Defending Vision Language Model Against Jailbreakscited by 44
  6. 2024AnyAttack: Towards Large-scale Self-supervised Adversarial Attacks on Vision-language Modelscited by 43
  7. 2024Immune: Improving Safety Against Jailbreaks in Multi-modal LLMs via Inference-Time Alignmentcited by 42
  8. 2025Jailbreaking Multimodal Large Language Models via Shuffle Inconsistencycited by 42
  9. 2024A Survey of Recent Backdoor Attacks and Defenses in Large Language Modelscited by 36
  10. 2024SafeSora: Towards Safety Alignment of Text2Video Generation via a Human Preference Datasetcited by 34
  11. 2024IDEATOR: Jailbreaking and Benchmarking Large Vision-Language Models Using Themselvescited by 34
  12. 2024Jailbreaking and Mitigation of Vulnerabilities in Large Language Modelscited by 33
  13. 2024ART: Automatic Red-teaming for Text-to-Image Models to Protect Benign Userscited by 32
  14. 2025Towards Robust and Secure Embodied AI: A Survey on Vulnerabilities and Attackscited by 32
  15. 2024Jailbreak Attacks and Defenses against Multimodal Generative Models: A Surveycited by 31
  16. 2025Distraction is All You Need for Multimodal Large Language Model Jailbreakingcited by 31
  17. 2024JailbreakEval: An Integrated Toolkit for Evaluating Jailbreak Attempts Against Large Language Modelscited by 28
  18. 2024BlueSuffix: Reinforced Blue Teaming for Vision-Language Models Against Jailbreak Attackscited by 28
  19. 2025Towards Understanding the Safety Boundaries of DeepSeek Models: Evaluation and Findingscited by 24
  20. 2024Cross-Modal Safety Alignment: Is textual unlearning all you need?cited by 22
  21. 2024Adversarial Attacks of Vision Tasks in the Past 10 Years: A Surveycited by 22
  22. 2024The Wolf Within: Covert Injection of Malice into MLLM Societies via an MLLM Operativecited by 21
  23. 2024Unbridled Icarus: A Survey of the Potential Perils of Image Inputs in Multimodal Large Language Model Securitycited by 18
  24. 2025NeuroStrike: Neuron-Level Attacks on Aligned LLMscited by 18
  25. 2024VLMGuard: Bootstrapping Malicious Prompt Detectors from Unlabeled Vision-Language Prompts in the Wildcited by 17
  26. 2025Understanding and Rectifying Safety Perception Distortion in VLMscited by 17
  27. 2025T2VShield: Model-Agnostic Jailbreak Defense for Text-to-Video Modelscited by 17
  28. 2025VLM-Guard: Safeguarding Vision-Language Models via Fulfilling Safety Alignment Gapcited by 16
  29. 2025Fun-tuning: Characterizing the Vulnerability of Proprietary LLMs to Optimization-based Prompt Injection Attacks via the Fine-Tuning Interfacecited by 15
  30. 2025Mirage in the Eyes: Hallucination Attack on Multi-modal Large Language Models with Only Attention Sinkcited by 15
  31. 2024The Synergy between Data and Multi-Modal Large Language Models: A Survey from Co-Development Perspectivecited by 14
  32. 2025Survey of Adversarial Robustness in Multimodal Large Language Modelscited by 14
  33. 2024Securing Vision-Language Models with a Robust Encoder Against Jailbreak and Adversarial Attackscited by 13
  34. 2024LeakAgent: RL-based Red-teaming Agent for LLM Privacy Leakagecited by 13
  35. 2025Attention! Your Vision Language Model Could Be Maliciously Manipulatedcited by 12
  36. 2024The Responsible Foundation Model Development Cheatsheet: A Review of Tools & Resourcescited by 11
  37. 2024Exploring Visual Vulnerabilities via Multi-Loss Adversarial Search for Jailbreaking Vision-Language Modelscited by 11
  38. 2025VisCRA: A Visual Chain Reasoning Attack for Jailbreaking Multimodal Large Language Modelscited by 11

50 most cited of 128 All 128 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Efficient LLM-Jailbreaking via Multimodal-LLM Jailbreak
  2. 2024JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attacks
  3. 2024textitMMJ-Bench: A Comprehensive Study on Jailbreak Attacks and Defenses for Multimodal Large Language Models
  4. 2024From LLMs to MLLMs: Exploring the Landscape of Multimodal Jailbreaking
  5. 2026Multi-turn Jailbreaking Attack in Multi-Modal Large Language Models
  6. 2025JPS: Jailbreak Multimodal Large Language Models with Collaborative Visual Perturbation and Textual Steering
  7. 2026DMN: A Compositional Framework for Jailbreaking Multimodal LLMs with Multi-Image Inputs
  8. 2025JailDAM: Jailbreak Detection with Adaptive Memory for Vision-Language Model