# Survey of Adversarial Robustness in Multimodal Large Language Models

arXiv: https://arxiv.org/abs/2503.13962 · HTML: https://mlsanity.com/paper/2503.13962/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 52 of its 61 references that are in the map.
- First posted on arXiv 18 Mar 2025.

## Placed in

- Multimodal LLMs / vision-language models / [MLLM jailbreaks and adversarial robustness](https://mlsanity.com/mllm/mllm-jailbreaks-and-adversarial-robustness/)

## Builds on (most cited first, 30 of 52)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering](https://mlsanity.com/paper/1612.00837/) (2016)
- [Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering](https://mlsanity.com/paper/2209.09513/) (2022)
- [Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models](https://mlsanity.com/paper/1505.04870/) (2015)
- [Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!](https://mlsanity.com/paper/2310.03693/) (2023)
- [OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledge](https://mlsanity.com/paper/1906.00067/) (2019)
- [Vision-Language Models for Vision Tasks: A Survey](https://mlsanity.com/paper/2304.00685/) (2023)
- [FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts](https://mlsanity.com/paper/2311.05608/) (2023)
- [Visual Adversarial Examples Jailbreak Aligned Large Language Models](https://mlsanity.com/paper/2306.13213/) (2023)
- [On Evaluating Adversarial Robustness of Large Vision-Language Models](https://mlsanity.com/paper/2305.16934/) (2023)
- [MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models](https://mlsanity.com/paper/2311.17600/) (2023)
- [SafetyBench: Evaluating the Safety of Large Language Models](https://mlsanity.com/paper/2309.07045/) (2023)
- [How Robust is Google's Bard to Adversarial Image Attacks?](https://mlsanity.com/paper/2309.11751/) (2023)
- [Image Hijacks: Adversarial Images can Control Generative Models at Runtime](https://mlsanity.com/paper/2309.00236/) (2023)
- [Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fast](https://mlsanity.com/paper/2402.08567/) (2024)
- [On the Adversarial Robustness of Multi-Modal Foundation Models](https://mlsanity.com/paper/2308.10741/) (2023)
- [Images are Achilles' Heel of Alignment: Exploiting Visual Vulnerabilities for Jailbreaking Multimodal Large Language Models](https://mlsanity.com/paper/2403.09792/) (2024)
- [Jailbreaking Attack against Multimodal Large Language Model](https://mlsanity.com/paper/2402.02309/) (2024)
- [LLaVA-Phi: Efficient Multi-Modal Assistant with Small Language Model](https://mlsanity.com/paper/2401.02330/) (2024)
- [Abusing Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMs](https://mlsanity.com/paper/2307.10490/) (2023)
- [Robust CLIP: Unsupervised Adversarial Fine-Tuning of Vision Embeddings for Robust Large Vision-Language Models](https://mlsanity.com/paper/2402.12336/) (2024)
- [Dissecting Adversarial Robustness of Multimodal LM Agents](https://mlsanity.com/paper/2406.12814/) (2024)
- [MLLM-Protector: Ensuring MLLM's Safety without Hurting Performance](https://mlsanity.com/paper/2401.02906/) (2024)
- [On the Robustness of Large Multimodal Models Against Image Adversarial Attacks](https://mlsanity.com/paper/2312.03777/) (2023)
- [VL-Trojan: Multimodal Instruction Backdoor Attacks against Autoregressive Visual Language Models](https://mlsanity.com/paper/2402.13851/) (2024)
- [A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trends](https://mlsanity.com/paper/2407.07403/) (2024)
- [AV-Deepfake1M: A Large-Scale LLM-Driven Audio-Visual Deepfake Dataset](https://mlsanity.com/paper/2311.15308/) (2023)
- [Safety of Multimodal Large Language Models on Images and Texts](https://mlsanity.com/paper/2402.00357/) (2024)
- [Inducing High Energy-Latency of Large Vision-Language Models with Verbose Images](https://mlsanity.com/paper/2401.11170/) (2024)

## Built on by (most cited first, 16 of 16)

- [MLLMs are Deeply Affected by Modality Bias](https://mlsanity.com/paper/2505.18657/) (2025)
- [A Survey on Autonomy-Induced Security Risks in Large Model-Based Agents](https://mlsanity.com/paper/2506.23844/) (2025)
- [Flattery in Motion: Benchmarking and Analyzing Sycophancy in Video-LLMs](https://mlsanity.com/paper/2506.07180/) (2025)
- [MVI-Bench: A Comprehensive Benchmark for Evaluating Robustness to Misleading Visual Inputs in LVLMs](https://mlsanity.com/paper/2511.14159/) (2025)
- [Robustness in Large Language Models: A Survey of Mitigation Strategies and Evaluation Metrics](https://mlsanity.com/paper/2505.18658/) (2025)
- [StableToken: A Noise-Robust Semantic Speech Tokenizer for Resilient SpeechLLMs](https://mlsanity.com/paper/2509.22220/) (2025)
- [Contextual Image Attack: How Visual Context Exposes Multimodal Safety Vulnerabilities](https://mlsanity.com/paper/2512.02973/) (2025)
- [Beyond Visual Safety: Jailbreaking Multimodal Large Language Models for Harmful Image Generation via Semantic-Agnostic Inputs](https://mlsanity.com/paper/2601.15698/) (2026)
- [RemoteShield: Enable Robust Multimodal Large Language Models for Earth Observation](https://mlsanity.com/paper/2604.17243/) (2026)
- [Robust Diagram Reasoning: A Framework for Enhancing LVLM Performance on Visually Perturbed Scientific Diagrams](https://mlsanity.com/paper/2508.16972/) (2025)
- [LLaVAShield: Safeguarding Multimodal Multi-Turn Dialogues in Vision-Language Models](https://mlsanity.com/paper/2509.25896/) (2025)
- [DefenSee: Dissecting Threat from Sight and Text -- A Multi-View Defensive Pipeline for Multi-modal Jailbreaks](https://mlsanity.com/paper/2512.01185/) (2025)
- [Universal Adversarial Attacks against Closed-Source MLLMs via Target-View Routed Meta Optimization](https://mlsanity.com/paper/2601.23179/) (2026)
- [Adversarial Robustness in Zero-Shot Learning:An Empirical Study on Class and Concept-Level Vulnerabilities](https://mlsanity.com/paper/2512.18651/) (2025)
- [POPS: Recovering Unlearned Multi-Modality Knowledge in MLLMs with Prompt-Optimized Parameter Shaking](https://mlsanity.com/paper/2607.06649/) (2026)
- [Visual-Invariance-Augmented Feature Optimal Alignment for Transferable Adversarial Attacks against Closed-Source MLLMs](https://mlsanity.com/paper/2610.06977/) (2026)

## Abstract (arXiv)

Multimodal Large Language Models (MLLMs) have demonstrated exceptional performance in artificial intelligence by facilitating integrated understanding across diverse modalities, including text, images, video, audio, and speech. However, their deployment in real-world applications raises significant concerns about adversarial vulnerabilities that could compromise their safety and reliability. Unlike unimodal models, MLLMs face unique challenges due to the interdependencies among modalities, making them susceptible to modality-specific threats and cross-modal adversarial manipulations. This paper reviews the adversarial robustness of MLLMs, covering different modalities. We begin with an overview of MLLMs and a taxonomy of adversarial attacks tailored to each modality. Next, we review key datasets and evaluation metrics used to assess the robustness of MLLMs. After that, we provide an in-depth review of attacks targeting MLLMs across different modalities. Our survey also identifies critical challenges and suggests promising future research directions.