Data to 14 Oct 2026

On the Robustness of Large Multimodal Models Against Image Adversarial Attacks

arXiv 2312.03777 ↗First posted 6 Dec 2023cs.CVcited by 92

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

Abstract (arXiv)

Recent advances in instruction tuning have led to the development of State-of-the-Art Large Multimodal Models (LMMs). Given the novelty of these models, the impact of visual adversarial attacks on LMMs has not been thoroughly examined. We conduct a comprehensive study of the robustness of various LMMs against different adversarial attacks, evaluated across tasks including image classification, image captioning, and Visual Question Answer (VQA). We find that in general LMMs are not robust to visual adversarial inputs. However, our findings suggest that context provided to the model via prompts, such as questions in a QA pair helps to mitigate the effects of visual adversarial inputs. Notably, the LMMs evaluated demonstrated remarkable resilience to such attacks on the ScienceQA task with only an 8.10% drop in performance compared to their visual counterparts which dropped 99.73%. We also propose a new approach to real-world image classification which we term query decomposition. By incorporating existence queries into our input prompt we observe diminished attack effectiveness and improvements in image classification accuracy. This research highlights a previously under-explored facet of LMM robustness and sets the stage for future work aimed at strengthening the resilience of multimodal systems in adversarial environments.

Key facts. Cited by 92 papers in the map (data through 14 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 38 of its 48 references that are in the map. First posted on arXiv 6 Dec 2023.

Builds on

38 of 48 references.

Closest ancestors · 1

Same direction, newest first

2023Visual Adversarial Examples Jailbreak Aligned Large Language Models22 Jun

Other references · 37

2014Microsoft COCO: Common Objects in Contextcited by 17,608
2023Visual Instruction Tuningcited by 7,869
2014Explaining and Harnessing Adversarial Examplescited by 7,364
2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
2013Intriguing properties of neural networkscited by 5,155
2023Improved Baselines with Visual Instruction Tuningcited by 4,418
2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
2023Qwen Technical Reportcited by 3,344
2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
2022Scaling Instruction-Finetuned Language Modelscited by 3,024

25 more

2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,919
2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,729
2019Towards VQA Models That Can Readcited by 1,718
2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,451
2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,268
2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 906
2016Practical Black-Box Attacks against Machine Learningcited by 903
2023LISA: Reasoning Segmentation via Large Language Modelcited by 776
2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 722
2017One pixel attack for fooling deep neural networkscited by 582
2017Adversarial Examples for Evaluating Reading Comprehension Systemscited by 571
2016Delving into Transferable Adversarial Examples and Black-box Attackscited by 536
2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 498
2016Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samplescited by 446
2018Generating Natural Language Adversarial Examplescited by 424
2017Mitigating Adversarial Effects Through Randomizationcited by 383
2020Torchattacks: A PyTorch Repository for Adversarial Attackscited by 158
2021Adversarial VQA: A New Benchmark for Evaluating the Robustness of VQA Modelscited by 75
2021Deep Hash Distillation for Image Retrievalcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 22; 2025: 34; 2026: 36.

92 papers.

Sort:

By area:

2024Jailbreak Vision Language Models via Bi-Modal Adversarial Promptcited by 109
2024A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trendscited by 84
2024SafeBench: A Safety Evaluation Framework for Multimodal Large Language Modelscited by 69
2025Manipulating Multimodal Agents via Cross-Modal Prompt Injectioncited by 44
2024Test-Time Backdoor Attacks on Multimodal Large Language Modelscited by 40
2024Chain of Attack: On the Robustness of Vision-Language Models Against Transfer-Based Adversarial Attackscited by 39
2025Towards Robust and Secure Embodied AI: A Survey on Vulnerabilities and Attackscited by 34
2024MM-SpuBench: Towards Better Understanding of Spurious Biases in Multimodal LLMscited by 33
2024Unveiling the Ignorance of MLLMs: Seeing Clearly, Answering Incorrectlycited by 30
2024R-Bench: Are your Large Multimodal Model Robust to Real-world Corruptions?cited by 30
2024Break the Visual Perception: Adversarial Attacks Targeting Encoded Visual Tokens of Large Vision-Language Modelscited by 27
2024B-AVIBench: Towards Evaluating the Robustness of Large Vision-Language Model on Black-box Adversarial Visual-Instructionscited by 26

38 more

2024Adversarial Attacks of Vision Tasks in the Past 10 Years: A Surveycited by 23
2025How Do Vision-Language Models Process Conflicting Information Across Modalities?cited by 23
2025ADVEDM:Fine-grained Adversarial Attack against VLM-based Embodied Agentscited by 23
2024SceneTAP: Scene-Coherent Typographic Adversarial Planner against Vision-Language Models in Real-World Environmentscited by 22
2025Virtual Agent Economiescited by 21
2025HumaniBench: A Human-Centric Framework for Large Multimodal Models Evaluationcited by 19
2025Survey of Adversarial Robustness in Multimodal Large Language Modelscited by 16
2025Mirage in the Eyes: Hallucination Attack on Multi-modal Large Language Models with Only Attention Sinkcited by 15
2025On the Robustness of GUI Grounding Models Against Image Attackscited by 14
2025MIP against Agent: Malicious Image Patches Hijacking Multimodal OS Agentscited by 12
2024Exploiting Vision Encoder Vulnerabilities for Universal Adversarial Perturbations on Large Vision-Language Modelscited by 11
2025JailBound: Jailbreaking Internal Safety Boundaries of Vision-Language Modelscited by 10
2024A Review of Multimodal Explainable Artificial Intelligence: Past, Present and Futurecited by 9
2025Poison as Cure: Visual Noise for Mitigating Object Hallucinations in LVMscited by 9
2025EVA: Evolving Semantic Adversaries for Red-Teaming GUI Agents Against Environmental Injection Attackscited by 9
2025V-Attack: Targeting Disentangled Value Features for Controllable Adversarial Attacks on LVLMscited by 9
2025AdPO: Enhancing the Adversarial Robustness of Large Vision-Language Models with Preference Optimizationcited by 8
2025VEAttack: Downstream-agnostic Vision Encoder Attack against Large Vision Language Modelscited by 8
2025Transferability of Adversarial Attacks in Video-based MLLMs: A Cross-modal Image-to-Video Approachcited by 7
2024Enhancing LLM-based Autonomous Driving Agents to Mitigate Perception Attackscited by 6
2025Implicit Jailbreak Attacks via Cross-Modal Information Concealment on Vision-Language Modelscited by 6
2026On the Adversarial Robustness of Large Vision-Language Models under Visual Token Compressioncited by 6
2024Transfer Attack for Bad and Good: Explain and Boost Adversarial Transferability across Multimodal Large Language Modelscited by 5
2024Towards Adversarially Robust Vision-Language Models: Insights from Design Choices and Prompt Formatting Techniquescited by 5
2024AdvDreamer Unveils: Are Vision-Language Models Truly Ready for Real-World 3D Variations?cited by 5
2025Tracking the Copyright of Large Vision-Language Models through Parameter Learning Adversarial Imagescited by 5
2025Visual Instruction Bottleneck Tuningcited by 5
2025Enhancing Targeted Adversarial Attacks on Large Vision-Language Models via Intermediate Projectorcited by 5
2024Light-weight Fine-tuning Method for Defending Adversarial Noise in Pre-trained Medical Vision-Language Modelscited by 4
2024Seeing is Deceiving: Exploitation of Visual Pathways in Multi-Modal Language Modelscited by 4
2025Beyond Cross-Modal Alignment: Measuring and Leveraging Modality Gap in Vision-Language Modelscited by 4
2026Adversarial Attacks on Multimodal Large Language Models: A Comprehensive Surveycited by 4
2026Making MLLMs Blind: Adversarial Smuggling Attacks in MLLM Content Moderationcited by 4
2025Adversarial-Guided Diffusion for Multimodal LLM Attackscited by 3
2025HV-Attack: Hierarchical Visual Attack for Multimodal Retrieval Augmented Generationcited by 3
2026PA-Attack: Guiding Gray-Box Attacks on LVLM Vision Encoders with Prototypes and Attentioncited by 3
2026Turing Test on Screen: A Benchmark for Mobile GUI Agent Humanizationcited by 3
2026On the Adversarial Robustness of Multimodal LLM Judgescited by 3

50 most cited of 92 All 92 (JSON)

Similar, not cited

Close in wording, no citation either way