mlsanityData to 7 Oct 2026

Unbridled Icarus: A Survey of the Potential Perils of Image Inputs in Multimodal Large Language Model Security

arXiv 2404.05264 ↗First posted 8 Apr 2024cs.CR cs.CVcited by 18

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

Abstract (arXiv)

Multimodal Large Language Models (MLLMs) demonstrate remarkable capabilities that increasingly influence various aspects of our daily lives, constantly defining the new boundary of Artificial General Intelligence (AGI). Image modalities, enriched with profound semantic information and a more continuous mathematical nature compared to other modalities, greatly enhance the functionalities of MLLMs when integrated. However, this integration serves as a double-edged sword, providing attackers with expansive vulnerabilities to exploit for highly covert and harmful attacks. The pursuit of reliable AI systems like powerful MLLMs has emerged as a pivotal area of contemporary research. In this paper, we endeavor to demostrate the multifaceted risks associated with the incorporation of image modalities into MLLMs. Initially, we delineate the foundational components and training processes of MLLMs. Subsequently, we construct a threat model, outlining the security vulnerabilities intrinsic to MLLMs. Moreover, we analyze and summarize existing scholarly discourses on MLLMs' attack and defense mechanisms, culminating in suggestions for the future research on MLLM security. Through this comprehensive analysis, we aim to deepen the academic understanding of MLLM security challenges and propel forward the development of trustworthy MLLM systems.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 58 of its 73 references that are in the map. First posted on arXiv 8 Apr 2024.

Builds on

58 of 73 references.

Closest ancestors · 27

Same direction, newest first

  1. 2024Images are Achilles' Heel of Alignment: Exploiting Visual Vulnerabilities for Jailbreaking Multimodal Large Language Models14 Mar
  2. 2024An Image Is Worth 1000 Lies: Adversarial Transferability across Prompts on Vision-Language Models14 Mar
  3. 2024Eyes Closed, Safety On: Protecting Multimodal LLMs via Image-to-Text Transformation14 Mar
  4. 2024AdaShield: Safeguarding Multimodal Large Language Models from Structure-based Attack via Adaptive Shield Prompting14 Mar
  5. 2024ImgTrojan: Jailbreaking Vision-Language Models with ONE Image5 Mar
  6. 2024One Prompt Word is Enough to Boost Adversarial Robustness for Pre-trained Vision-Language Models4 Mar
  7. 2024VL-Trojan: Multimodal Instruction Backdoor Attacks against Autoregressive Visual Language Models21 Feb
  8. 2024Test-Time Backdoor Attacks on Multimodal Large Language Models13 Feb
  9. 2024Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fast13 Feb
  10. 2024Shadowcast: Stealthy Data Poisoning Attacks Against Vision-Language Models5 Feb
  11. 2024Vision-LLMs Can Fool Themselves with Self-Generated Typographic Attacks1 Feb
  12. 2024Red Teaming Visual Language Models23 Jan
15 more
  1. 2024MLLM-Protector: Ensuring MLLM's Safety without Hurting Performance5 Jan
  2. 2023SA-Attack: Improving Adversarial Transferability of Vision-Language Pre-training Models via Self-Augmentation8 Dec
  3. 2023OT-Attack: Enhancing Adversarial Transferability of Vision-Language Models via Optimal Transport Optimization7 Dec
  4. 2023InstructTA: Instruction-Tuned Targeted Attack for Large Vision-Language Models4 Dec
  5. 2023Adversarial Prompt Tuning for Vision-Language Models19 Nov
  6. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts9 Nov
  7. 2023How Robust is Google's Bard to Adversarial Image Attacks?21 Sep
  8. 2023Exploring Transferability of Multimodal Adversarial Samples for Vision-Language Pre-training Models with Contrastive Learning24 Aug
  9. 2023Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models26 Jul
  10. 2023Set-level Guidance Attack: Boosting Adversarial Transferability of Vision-Language Pre-training Models26 Jul
  11. 2023Abusing Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMs19 Jul
  12. 2023Visual Adversarial Examples Jailbreak Aligned Large Language Models22 Jun
  13. 2023On Evaluating Adversarial Robustness of Large Vision-Language Models26 May
  14. 2023Robust Contrastive Language-Image Pre-training against Data Poisoning and Backdoor Attacks13 Mar
  15. 2022Towards Adversarial Attack on Vision-Language Pre-training Models19 Jun

Other references · 31

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  2. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  3. 2024MM-LLMs: Recent Advances in MultiModal Large Language Modelscited by 277
  4. 2023AI Alignment: A Comprehensive Surveycited by 256
  5. 2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attackscited by 189
  6. 2023Knowledge Editing for Large Language Models: A Surveycited by 189
  7. 2023Image Hijacks: Adversarial Images can Control Generative Models at Runtimecited by 165
  8. 2024Eight Methods to Evaluate Robust Unlearning in LLMscited by 153
  9. 2023On the Adversarial Robustness of Multi-Modal Foundation Modelscited by 142
  10. 2024Jailbreaking Attack against Multimodal Large Language Modelcited by 128
  11. 2023AdvCLIP: Downstream-agnostic Adversarial Examples in Multimodal Contrastive Learningcited by 101
  12. 2023DRESS: Instructing Large Vision-Language Models to Align and Interact with Humans via Natural Language Feedbackcited by 99
19 more
  1. 2023A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validationcited by 91
  2. 2024Detoxifying Large Language Models via Knowledge Editingcited by 86
  3. 2021Hidden Backdoors in Human-Centric Language Modelscited by 83
  4. 2024InferAligner: Inference-Time Alignment for Harmlessness through Cross-Model Guidancecited by 78
  5. 2023VLATTACK: Multimodal Adversarial Attacks on Vision-Language Tasks via Pre-trained Modelscited by 66
  6. 2024Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systemscited by 63
  7. 2023Jailbreaking GPT-4V via Self-Adversarial Attacks with System Promptscited by 62
  8. 2023Right to be Forgotten in the Era of Large Language Models: Implications, Challenges, and Solutionscited by 59
  9. 2023A Survey on Federated Unlearning: Challenges, Methods, and Future Directionscited by 58
  10. 2024Learn What You Want to Unlearn: Unlearning Inversion Attacks against Machine Unlearningcited by 43
  11. 2024Threats, Attacks, and Defenses in Machine Unlearning: A Surveycited by 34
  12. 2023A Duty to Forget, a Right to be Assured? Exposing Vulnerabilities in Machine Unlearning Servicescited by 28
  13. 2024The First to Know: How Token Distributions Reveal Hidden Knowledge in Large Vision-Language Models?cited by 24
  14. 2024The Wolf Within: Covert Injection of Malice into MLLM Societies via an MLLM Operativecited by 21
  15. 2024On the Essence and Prospect: An Investigation of Alignment Approaches for Big Modelscited by 21
  16. 2024Editing Conceptual Knowledge for Large Language Modelscited by 21
  17. 2024Towards Efficient and Certified Recovery from Poisoning Attacks in Federated Learningcited by 7
  18. 2023Bounded and Unbiased Composite Differential Privacycited by 3
  19. 2024A Learning-based Incentive Mechanism for Mobile AIGC Service in Decentralized Internet of Vehiclescited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 11; 2026: 1.

18 papers.

Sort:

By area:

  1. 2024A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trendscited by 82
  2. 2025A Survey of Safety on Large Vision-Language Models: Attacks, Defenses and Evaluationscited by 66
  3. 2024Cross-Modal Safety Alignment: Is textual unlearning all you need?cited by 22
  4. 2024From Specific-MLLMs to Omni-MLLMs: A Survey on MLLMs Aligned with Multi-modalitiescited by 22
  5. 2024The Synergy between Data and Multi-Modal Large Language Models: A Survey from Co-Development Perspectivecited by 14
  6. 2025Survey of Adversarial Robustness in Multimodal Large Language Modelscited by 14
  7. 2025Is LLM an Overconfident Judge? Unveiling the Capabilities of LLMs in Detecting Offensive Language with Annotation Disagreementcited by 13
  8. 2025Pushing the Limits of Safety: A Technical Report on the ATLAS Challenge 2025cited by 11
  9. 2024From Text to Multimodality: Exploring the Evolution and Impact of Large Language Models in Medical Practicecited by 8
  10. 2025Improving Generalizability and Undetectability for Targeted Adversarial Attacks on Multimodal Pre-trained Modelscited by 5
  11. 2024Seeing is Deceiving: Exploitation of Visual Pathways in Multi-Modal Language Modelscited by 4
  12. 2025SoK: Systematizing LLM Prompt Security: Taxonomies, Datasets, and Unified Evaluation of Attacks and Defensescited by 4
6 more
  1. 2025Safeguarding Vision-Language Models: Mitigating Vulnerabilities to Gaussian Noise in Perturbation-based Attackscited by 3
  2. 2025Fighting Fire with Fire (F3): A Training-free and Efficient Visual Adversarial Example Purification Method in LVLMscited by 1
  3. 2025Learning to Detect Unknown Jailbreak Attacks in Large Vision-Language Modelscited by 1
  4. 2025An Enhanced Large Language Model For Cross Modal Query Understanding System Using DL-KeyBERT Based CAZSSCL-MPGPTcited by 0
  5. 2025Blind Spot Navigation: Evolutionary Discovery of Sensitive Semantic Concepts for LVLMscited by 0
  6. 2026Evolving Safety Landscape of Multi-modal Large Language Models: A Survey of Emerging Threats and Safeguardscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Adversarial Attacks on Multimodal Large Language Models: A Comprehensive Survey
  2. 2025Investigating Vulnerabilities and Defenses Against Audio-Visual Attacks: A Comprehensive Survey Emphasizing Multimodal Models
  3. 2024Safety of Multimodal Large Language Models on Images and Texts
  4. 2025Odysseus: Jailbreaking Commercial Multimodal LLM-integrated Systems via Dual Steganography
  5. 2023MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models
  6. 2026Leave My Images Alone: Preventing Multi-Modal Large Language Models from Analyzing Images via Visual Prompt Injection
  7. 2025Q-MLLM: Vector Quantization for Robust Multimodal Large Language Model Security
  8. 2026SafeSteer: A Decoding-level Defense Mechanism for Multimodal Large Language Models