mlsanityData to 7 Oct 2026

Jailbreak Vision Language Models via Bi-Modal Adversarial Prompt

arXiv 2406.04031 ↗First posted 6 Jun 2024cs.CV cs.CRcited by 103

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

also in 1 more
Abstract (arXiv)

In the realm of large vision language models (LVLMs), jailbreak attacks serve as a red-teaming approach to bypass guardrails and uncover safety implications. Existing jailbreaks predominantly focus on the visual modality, perturbing solely visual inputs in the prompt for attacks. However, they fall short when confronted with aligned models that fuse visual and textual features simultaneously for generation. To address this limitation, this paper introduces the Bi-Modal Adversarial Prompt Attack (BAP), which executes jailbreaks by optimizing textual and visual prompts cohesively. Initially, we adversarially embed universally harmful perturbations in an image, guided by a few-shot query-agnostic corpus (e.g., affirmative prefixes and negative inhibitions). This process ensures that image prompt LVLMs to respond positively to any harmful queries. Subsequently, leveraging the adversarial image, we optimize textual prompts with specific harmful intent. In particular, we utilize a large language model to analyze jailbreak failures and employ chain-of-thought reasoning to refine textual prompts through a feedback-iteration manner. To validate the efficacy of our approach, we conducted extensive evaluations on various datasets and LVLMs, demonstrating that our method significantly outperforms other methods by large margins (+29.03% in attack success rate on average). Additionally, we showcase the potential of our attacks on black-box commercial LVLMs, such as Gemini and ChatGLM.

Key facts. Cited by 103 papers in the map (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 69 of its 90 references that are in the map. First posted on arXiv 6 Jun 2024.

Builds on

69 of 90 references.

Closest ancestors · 14

Same direction, newest first

  1. 2024An Image Is Worth 1000 Lies: Adversarial Transferability across Prompts on Vision-Language Models14 Mar
  2. 2024Eyes Closed, Safety On: Protecting Multimodal LLMs via Image-to-Text Transformation14 Mar
  3. 2024ImgTrojan: Jailbreaking Vision-Language Models with ONE Image5 Mar
  4. 2024VL-Trojan: Multimodal Instruction Backdoor Attacks against Autoregressive Visual Language Models21 Feb
  5. 2024Vision-LLMs Can Fool Themselves with Self-Generated Typographic Attacks1 Feb
  6. 2024MLLM-Protector: Ensuring MLLM's Safety without Hurting Performance5 Jan
  7. 2023SA-Attack: Improving Adversarial Transferability of Vision-Language Pre-training Models via Self-Augmentation8 Dec
  8. 2023On the Robustness of Large Multimodal Models Against Image Adversarial Attacks6 Dec
  9. 2023InstructTA: Instruction-Tuned Targeted Attack for Large Vision-Language Models4 Dec
  10. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts9 Nov
  11. 2023How Robust is Google's Bard to Adversarial Image Attacks?21 Sep
  12. 2023Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models26 Jul
2 more
  1. 2023Visual Adversarial Examples Jailbreak Aligned Large Language Models22 Jun
  2. 2023On Evaluating Adversarial Robustness of Large Vision-Language Models26 May

Other references · 55

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  6. 2023Visual Instruction Tuningcited by 7,792
  7. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  8. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  9. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  10. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  11. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  12. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
38 more
  1. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  2. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  3. 2023LIMA: Less Is More for Alignmentcited by 1,080
  4. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 738
  5. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 703
  6. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 619
  7. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 520
  8. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 434
  9. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrationscited by 401
  10. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreakercited by 337
  11. 2023Are aligned neural networks adversarially aligned?cited by 305
  12. 2021Image Retrieval on Real-life Images with Pre-trained Vision-and-Language Modelscited by 244
  13. 2023Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulationcited by 202
  14. 2021VisualGPT: Data-efficient Adaptation of Pretrained Language Models for Image Captioningcited by 190
  15. 2023Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Futurecited by 184
  16. 2024WARM: On the Benefits of Weight Averaged Reward Modelscited by 145
  17. 2023On the Adversarial Robustness of Multi-Modal Foundation Modelscited by 142
  18. 2024Jailbreaking Attack against Multimodal Large Language Modelcited by 128
  19. 2021Dual Attention Suppression Attack: Generate Adversarial Camouflage in Physical Worldcited by 127
  20. 2023BadCLIP: Dual-Embedding Guided Backdoor Attack on Multimodal Contrastive Learningcited by 112
  21. 2018Transferable Adversarial Attacks for Image and Video Object Detectioncited by 109
  22. 2023DRESS: Instructing Large Vision-Language Models to Align and Interact with Humans via Natural Language Feedbackcited by 99
  23. 2022Parallel Rectangle Flip Attack: A Query-based Black-box Attack against Object Detectioncited by 79
  24. 2020Efficient Adversarial Attacks for Visual Object Trackingcited by 66
  25. 2023X-Adv: Physical Adversarial Object Attacks against X-ray Prohibited Item Detectioncited by 58
  26. 2022A Large-scale Multiple-objective Method for Black-box Attack against Object Detectioncited by 55
  27. 2024Semantic Mirror Jailbreak: Genetic Algorithm Based Jailbreak Prompts Against Open-source LLMscited by 48
  28. 2023Vision Transformer with Quadrangle Attentioncited by 46
  29. 2023JailGuard: A Universal Detection Framework for LLM Prompt-based Attackscited by 46
  30. 2021Harnessing Perceptual Adversarial Patches for Crowd Countingcited by 39
  31. 2023Pre-trained Trojan Attacks for Visual Recognitioncited by 38
  32. 2023Diversifying the High-level Features for better Adversarial Transferabilitycited by 34
  33. 2024Poisoned Forgery Face: Towards Backdoor Attacks on Face Forgery Detectioncited by 34
  34. 2024AttackEval: How to Evaluate the Effectiveness of Jailbreak Attacking on Large Language Modelscited by 26
  35. 2023Fool Your (Vision and) Language Model With Embarrassingly Simple Permutationscited by 24
  36. 2024Hide in Thicket: Generating Imperceptible and Rational Adversarial Perturbations on 3D Point Cloudscited by 24
  37. 2023Improving Adversarial Transferability by Stable Diffusioncited by 22
  38. 2022Exploring Inconsistent Knowledge Distillation for Object Detection with Data Augmentationcited by 19

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 17; 2025: 48; 2026: 38.

103 papers.

Sort:

By area:

  1. 2025A Survey of Safety on Large Vision-Language Models: Attacks, Defenses and Evaluationscited by 66
  2. 2025Reasoning-Augmented Conversation for Multi-Turn Jailbreak Attacks on Large Language Modelscited by 61
  3. 2025A Survey of LLM-Driven AI Agent Communication: Protocols, Security Risks, and Defense Countermeasurescited by 54
  4. 2024VHELM: A Holistic Evaluation of Vision Language Modelscited by 53
  5. 2024Visual Prompting in Multimodal Large Language Models: A Surveycited by 52
  6. 2024AnyAttack: Towards Large-scale Self-supervised Adversarial Attacks on Vision-language Modelscited by 43
  7. 2024Immune: Improving Safety Against Jailbreaks in Multi-modal LLMs via Inference-Time Alignmentcited by 42
  8. 2025Manipulating Multimodal Agents via Cross-Modal Prompt Injectioncited by 42
  9. 2024Unveiling the Safety of GPT-4o: An Empirical Study using Jailbreak Attackscited by 35
  10. 2024Visual Adversarial Attack on Vision-Language Models for Autonomous Drivingcited by 35
  11. 2025AGENTSAFE: Benchmarking the Safety of Embodied Agents on Hazardous Instructionscited by 33
  12. 2024Audio Is the Achilles' Heel: Red Teaming Audio Large Multimodal Modelscited by 31
38 more
  1. 2024Jailbreak Attacks and Defenses against Multimodal Generative Models: A Surveycited by 31
  2. 2024Heuristic-Induced Multimodal Risk Distribution Jailbreak Attack for Multimodal Large Language Modelscited by 29
  3. 2024BlueSuffix: Reinforced Blue Teaming for Vision-Language Models Against Jailbreak Attackscited by 28
  4. 2025Towards Understanding the Safety Boundaries of DeepSeek Models: Evaluation and Findingscited by 24
  5. 2024Adversarial Attacks of Vision Tasks in the Past 10 Years: A Surveycited by 22
  6. 2025T2V-OptJail: Discrete Prompt Optimization for Text-to-Video Jailbreak Attackscited by 18
  7. 2024PBI-Attack: Prior-Guided Bimodal Interactive Black-Box Jailbreak Attack for Toxicity Maximizationcited by 17
  8. 2025T2VShield: Model-Agnostic Jailbreak Defense for Text-to-Video Modelscited by 17
  9. 2025SecureWebArena: A Holistic Security Evaluation Benchmark for LVLM-based Web Agentscited by 17
  10. 2025Breaking the Ceiling: Exploring the Potential of Jailbreak Attacks through Expanding Strategy Spacecited by 15
  11. 2025SafeMobile: Chain-level Jailbreak Detection and Automated Evaluation for Multimodal Mobile Agentscited by 14
  12. 2025E2AT: Multimodal Jailbreak Defense via Dynamic Joint Optimization for Multimodal Large Language Modelscited by 12
  13. 2025Attention! Your Vision Language Model Could Be Maliciously Manipulatedcited by 12
  14. 2024Exploring Visual Vulnerabilities via Multi-Loss Adversarial Search for Jailbreaking Vision-Language Modelscited by 11
  15. 2025Pushing the Limits of Safety: A Technical Report on the ATLAS Challenge 2025cited by 11
  16. 2025RoboSafe: Safeguarding Embodied Agents via Executable Safety Logiccited by 11
  17. 2026When World Models Dream Wrong: Physical-Conditioned Adversarial Attacks against World Modelscited by 11
  18. 2025OmniSafeBench-MM: A Unified Benchmark and Toolbox for Multimodal Jailbreak Attack-Defense Evaluationcited by 10
  19. 2024Safe + Safe = Unsafe? Exploring How Safe Images Can Be Exploited to Jailbreak Large Vision-Language Modelscited by 9
  20. 2025Detoxifying Large Language Models via Autoregressive Reward Guided Representation Editingcited by 9
  21. 2025Three Minds, One Legend: Jailbreak Large Reasoning Model with Adaptive Stacked Cipherscited by 8
  22. 2025Align is not Enough: Multimodal Universal Jailbreak Attack against Multimodal Large Language Modelscited by 8
  23. 2025"I am bad": Interpreting Stealthy, Universal and Robust Audio Jailbreaks in Audio-Language Modelscited by 7
  24. 2025POISONCRAFT: Practical Poisoning of Retrieval-Augmented Generation for Large Language Modelscited by 6
  25. 2025Mask-GCG: Are All Tokens in Adversarial Suffixes Necessary for Jailbreak Attacks?cited by 6
  26. 2025Odysseus: Jailbreaking Commercial Multimodal LLM-integrated Systems via Dual Steganographycited by 6
  27. 2026CtrlAttack: A Unified Attack on World-Model Control in Diffusion Modelscited by 6
  28. 2026JailWAM: Jailbreaking World Action Models in Robot Controlcited by 5
  29. 2025Making Every Step Effective: Jailbreaking Large Vision-Language Models Through Hierarchical KV Equalizationcited by 4
  30. 2025REVEAL: Multi-turn Evaluation of Image-Input Harms for Vision LLMcited by 4
  31. 2025No Query, No Accesscited by 4
  32. 2025Hierarchical Safety Realignment: Lightweight Restoration of Safety in Pruned Large Vision-Language Modelscited by 4
  33. 2025SafeLLM: Unlearning Harmful Outputs from Large Language Models against Jailbreak Attackscited by 4
  34. 2025On Surjectivity of Neural Networks: Can you elicit any behavior from your model?cited by 4
  35. 2026Tone Matters: The Impact of Linguistic Tone on Hallucination in VLMscited by 4
  36. 2024LIAR: Leveraging Inference Time Alignment (Best-of-N) to Jailbreak LLMs in Secondscited by 3
  37. 2025The Safety Reminder: A Soft Prompt to Reactivate Delayed Safety Awareness in Vision-Language Modelscited by 3
  38. 2025PRISM: Programmatic Reasoning with Image Sequence Manipulation for LVLM Jailbreakingcited by 3

50 most cited of 103 All 103 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Toward Universal and Transferable Jailbreak Attacks on Vision-Language Models
  2. 2025Cross-Modal Obfuscation for Jailbreak Attacks on Large Vision-Language Models
  3. 2025Towards Effective MLLM Jailbreaking Through Balanced On-Topicness and OOD-Intensity
  4. 2025Implicit Jailbreak Attacks via Cross-Modal Information Concealment on Vision-Language Models
  5. 2025TAIJI: Textual Anchoring for Immunizing Jailbreak Images in Vision Language Models
  6. 2025VERA-V: Variational Inference Framework for Jailbreaking Vision-Language Models
  7. 2024White-box Multimodal Jailbreaks Against Large Vision-Language Models
  8. 2024Deciphering the Chaos: Enhancing Jailbreak Attacks via Adversarial Prompt Translation