mlsanityData to 7 Oct 2026

Universal Adversarial Attack on Aligned Multimodal LLMs

arXiv 2502.07987 ↗First posted 11 Feb 2025cs.AIcited by 5

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

Abstract (arXiv)

We propose a universal adversarial attack on multimodal Large Language Models (LLMs) that leverages a single optimized image to override alignment safeguards across diverse queries and even multiple models. By backpropagating through the vision encoder and language head, we craft a synthetic image that forces the model to respond with a targeted phrase (e.g., "Sure, here it is") or otherwise unsafe content -- even for harmful prompts. In experiments on the SafeBench and MM-SafetyBench benchmarks, our method achieves higher attack success rates than existing baselines, including text-only universal prompts (e.g., up to 81% on certain models). We further demonstrate cross-model universality by training on several multimodal LLMs simultaneously. Additionally, a multi-answer variant of our approach produces more natural-sounding (yet still malicious) responses. These findings underscore critical vulnerabilities in current multimodal alignment and call for more robust adversarial defenses. We will release code and datasets under the Apache-2.0 license. Warning: some content generated by Multimodal LLMs in this paper may be offensive.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 24 of its 35 references that are in the map. First posted on arXiv 11 Feb 2025.

Builds on

24 of 35 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024Llama Guard 3 Vision: Safeguarding Human-AI Image Understanding Conversations15 Nov
  2. 2024Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fast13 Feb
  3. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts9 Nov
  4. 2022Towards Adversarial Attack on Vision-Language Pre-training Models19 Jun

Other references · 20

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2013Intriguing properties of neural networkscited by 5,114
  3. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  4. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  5. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  6. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  7. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  8. 2016Adversarial examples in the physical worldcited by 1,762
  9. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  10. 2016Universal adversarial perturbationscited by 774
  11. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
  12. 2023MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Actioncited by 511
8 more
  1. 2016Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samplescited by 439
  2. 2018TextBugger: Generating Adversarial Text Against Real-world Applicationscited by 376
  3. 2023Are aligned neural networks adversarially aligned?cited by 305
  4. 2021Gradient-based Adversarial Attacks against Text Transformerscited by 233
  5. 2023Automatically Auditing Large Language Models via Discrete Optimizationcited by 187
  6. 2023VLATTACK: Multimodal Adversarial Attacks on Vision-Language Tasks via Pre-trained Modelscited by 66
  7. 2024SafeBench: A Safety Evaluation Framework for Multimodal Large Language Modelscited by 63
  8. 2021Can audio-visual integration strengthen robustness under multimodal attacks?cited by 23

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 2.

5 papers.

  1. 2025Security Challenges in AI Agent Deployment: Insights from a Large Scale Public Competitioncited by 22
  2. 2025Adversarial Confusion Attack: Disrupting Multimodal Large Language Modelscited by 3
  3. 2026Diagnosing and Repairing Unsafe Channels in Vision-Language Models via Causal Discovery and Dual-Modal Safety Subspace Projectioncited by 1
  4. 2025Breaking the Illusion: Consensus-Based Generative Mitigation of Adversarial Illusions in Multi-Modal Embeddingscited by 0
  5. 2026VisInject: Disruption != Injection -- A Dual-Dimension Evaluation of Universal Adversarial Attacks on Vision-Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Adversarial Attacks on Multimodal Large Language Models: A Comprehensive Survey
  2. 2026Alignment Drift in Multimodal LLMs: A Two-Phase, Longitudinal Evaluation of Harm Across Eight Model Releases
  3. 2023Visual Adversarial Examples Jailbreak Aligned Large Language Models
  4. 2025Survey of Adversarial Robustness in Multimodal Large Language Models
  5. 2023Defending Against Alignment-Breaking Attacks via Robustly Aligned LLM
  6. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Models
  7. 2025Align is not Enough: Multimodal Universal Jailbreak Attack against Multimodal Large Language Models
  8. 2023Misusing Tools in Large Language Models With Visual Adversarial Examples