mlsanityData to 7 Oct 2026

Safety of Multimodal Large Language Models on Images and Texts

arXiv 2402.00357 ↗First posted 1 Feb 2024cs.CVcited by 70

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

Abstract (arXiv)

Attracted by the impressive power of Multimodal Large Language Models (MLLMs), the public is increasingly utilizing them to improve the efficiency of daily work. Nonetheless, the vulnerabilities of MLLMs to unsafe instructions bring huge safety risks when these models are deployed in real-world scenarios. In this paper, we systematically survey current efforts on the evaluation, attack, and defense of MLLMs' safety on images and text. We begin with introducing the overview of MLLMs on images and text and understanding of safety, which helps researchers know the detailed scope of our survey. Then, we review the evaluation datasets and metrics for measuring the safety of MLLMs. Next, we comprehensively present attack and defense techniques related to MLLMs' safety. Finally, we analyze several unsolved issues and discuss promising research directions. The latest papers are continually collected at https://github.com/isXinLiu/MLLM-Safety-Collection.

Key facts. Cited by 70 papers in the map (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 38 of its 44 references that are in the map. First posted on arXiv 1 Feb 2024.

Builds on

38 of 44 references.

Closest ancestors · 9

Same direction, newest first

  1. 2024Red Teaming Visual Language Models23 Jan
  2. 2024MLLM-Protector: Ensuring MLLM's Safety without Hurting Performance5 Jan
  3. 2023MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models29 Nov
  4. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts9 Nov
  5. 2023Can Language Models be Instructed to Protect Personal Information?3 Oct
  6. 2023How Robust is Google's Bard to Adversarial Image Attacks?21 Sep
  7. 2023Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models26 Jul
  8. 2023Abusing Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMs19 Jul
  9. 2023Visual Adversarial Examples Jailbreak Aligned Large Language Models22 Jun

Other references · 29

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  4. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  5. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  6. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 828
  7. 2020The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memescited by 485
  8. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 477
  9. 2023Are aligned neural networks adversarially aligned?cited by 305
  10. 2024TrustLLM: Trustworthiness in Large Language Modelscited by 280
  11. 2023Can Pre-trained Vision and Language Models Answer Visual Information-Seeking Questions?cited by 203
  12. 2023SafetyBench: Evaluating the Safety of Large Language Modelscited by 189
17 more
  1. 2021Challenges in Detoxifying Language Modelscited by 169
  2. 2023Image Hijacks: Adversarial Images can Control Generative Models at Runtimecited by 165
  3. 2023On the Adversarial Robustness of Multi-Modal Foundation Modelscited by 142
  4. 2023How Many Unicorns Are in This Image? A Safety Evaluation Benchmark for Vision LLMscited by 103
  5. 2023DRESS: Instructing Large Vision-Language Models to Align and Interact with Humans via Natural Language Feedbackcited by 99
  6. 2023Safety Assessment of Chinese Large Language Modelscited by 82
  7. 2024InferAligner: Inference-Time Alignment for Harmlessness through Cross-Model Guidancecited by 78
  8. 2021Detecting Harmful Memes and Their Targetscited by 77
  9. 2023Jailbreaking GPT-4V via Self-Adversarial Attacks with System Promptscited by 62
  10. 2024GOAT-Bench: Safety Insights to Large Multimodal Models through Meme-Based Social Abusecited by 50
  11. 2023Misusing Tools in Large Language Models With Visual Adversarial Examplescited by 40
  12. 2021Bridge the Gap Between CV and NLP! A Gradient-based Textual Adversarial Attack Frameworkcited by 27
  13. 2023ToViLaG: Your Visual-Language Generative Model is Also An Evildoercited by 27
  14. 2017Protest Activity Detection and Perceived Violence Estimation from Social Media Imagescited by 20
  15. 2023Detecting and Correcting Hate Speech in Multimodal Memes with Large Visual Language Modelcited by 12
  16. 2023Large Language Models as Automated Aligners for benchmarking Vision-Language Modelscited by 10
  17. 2023Adventures of Trustworthy Vision-Language Models: A Surveycited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 16; 2025: 36; 2026: 18.

70 papers.

Sort:

By area:

  1. 2024Eyes Closed, Safety On: Protecting Multimodal LLMs via Image-to-Text Transformationcited by 114
  2. 2024AdaShield: Safeguarding Multimodal Large Language Models from Structure-based Attack via Adaptive Shield Promptingcited by 113
  3. 2024A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trendscited by 82
  4. 2025A Survey of Safety on Large Vision-Language Models: Attacks, Defenses and Evaluationscited by 66
  5. 2024ETA: Evaluating Then Aligning Safety of Vision Language Models at Inference Timecited by 57
  6. 2024Visual Prompting in Multimodal Large Language Models: A Surveycited by 52
  7. 2025Adversarial Attacks against Closed-Source MLLMs via Feature Optimal Alignmentcited by 42
  8. 2024Align Anything: Training All-Modality Models to Follow Instructions with Language Feedbackcited by 38
  9. 2024Assessment of Multimodal Large Language Models in Alignment with Human Valuescited by 36
  10. 2025A Survey on Mechanistic Interpretability for Multi-Modal Foundation Modelscited by 34
  11. 2025SafeEraser: Enhancing Safety in Multimodal Large Language Models through Multimodal Machine Unlearningcited by 31
  12. 2025HiddenDetect: Detecting Jailbreak Attacks against Large Vision-Language Models via Monitoring Hidden Statescited by 29
38 more
  1. 2025Safe RLHF-V: Safe Reinforcement Learning from Multi-modal Human Feedbackcited by 27
  2. 2024Safe Inputs but Unsafe Output: Benchmarking Cross-modality Safety Alignment of Large Vision-Language Modelcited by 24
  3. 2024The Wolf Within: Covert Injection of Malice into MLLM Societies via an MLLM Operativecited by 21
  4. 2024RapGuard: Safeguarding Multimodal Large Language Models via Rationale-aware Defensive Promptingcited by 19
  5. 2025Safety Mirage: How Spurious Correlations Undermine VLM Safety Fine-Tuning and Can Be Mitigated by Machine Unlearningcited by 16
  6. 2025Survey of Adversarial Robustness in Multimodal Large Language Modelscited by 14
  7. 2025Robust-LLaVA: On the Effectiveness of Large-Scale Robust Image Encoders for Multi-modal Large Language Modelscited by 9
  8. 2025MLLMEraser: Achieving Test-Time Unlearning in Multimodal Large Language Models through Activation Steeringcited by 9
  9. 2024Effective and Efficient Adversarial Detection for Vision-Language Models via A Single Vectorcited by 8
  10. 2025MIRAGE: Multimodal Immersive Reasoning and Guided Exploration for Red-Team Jailbreak Attackscited by 8
  11. 2025AdPO: Enhancing the Adversarial Robustness of Large Vision-Language Models with Preference Optimizationcited by 8
  12. 2025SaFeR-VLM: Toward Safety-aware Fine-grained Reasoning in Multimodal Modelscited by 8
  13. 2025Spot Risks Before Speaking! Unraveling Safety Attention Heads in Large Vision-Language Modelscited by 7
  14. 2024Zer0-Jack: A Memory-efficient Gradient-based Jailbreaking Method for Black-box Multi-modal Large Language Modelscited by 6
  15. 2025Steering Multimodal Large Language Models Decoding for Context-Aware Safetycited by 6
  16. 2025Medical Multimodal Model Stealing Attacks via Adversarial Domain Alignmentcited by 5
  17. 2025Improving Generalizability and Undetectability for Targeted Adversarial Attacks on Multimodal Pre-trained Modelscited by 5
  18. 2026Evolving Contextual Safety in Multi-Modal Large Language Models via Inference-Time Self-Reflective Memorycited by 5
  19. 2024Seeing is Deceiving: Exploitation of Visual Pathways in Multi-Modal Language Modelscited by 4
  20. 2025REVEAL: Multi-turn Evaluation of Image-Input Harms for Vision LLMcited by 4
  21. 2025TRAP: Targeted Redirecting of Agentic Preferencescited by 4
  22. 2025Failures to Surface Harmful Contents in Video Large Language Modelscited by 4
  23. 2025Unveiling Trust in Multimodal Large Language Models: Evaluation, Analysis, and Mitigationcited by 4
  24. 2026Unveiling Privacy Risks in Multi-modal Large Language Models: Task-specific Vulnerabilities and Mitigation Challengescited by 4
  25. 2025The Safety Reminder: A Soft Prompt to Reactivate Delayed Safety Awareness in Vision-Language Modelscited by 3
  26. 2026PII-VisBench: Evaluating Personally Identifiable Information Safety in Vision Language Models Along a Continuum of Visibilitycited by 3
  27. 2026Pragma-VL: Towards a Pragmatic Arbitration of Safety and Helpfulness in MLLMscited by 3
  28. 2026Visual-Noise Guided In-Context Distillation for Multimodal Large Language Model Unlearningcited by 3
  29. 2024Embedding an Ethical Mind: Aligning Text-to-Image Synthesis via Lightweight Value Optimizationcited by 2
  30. 2025Probabilistic Modeling of Jailbreak on Multimodal LLMs: From Quantification to Applicationcited by 2
  31. 2025SmoothGuard: Defending Multimodal Large Language Models with Noise Perturbation and Clustering Aggregationcited by 2
  32. 2025Q-MLLM: Vector Quantization for Robust Multimodal Large Language Model Securitycited by 2
  33. 2026Risk Awareness Injection: Calibrating Vision-Language Models for Safety without Compromising Utilitycited by 2
  34. 2026Visual Self-Fulfilling Alignment: Shaping Safety-Oriented Personas via Threat-Related Imagescited by 2
  35. 2026When Understanding Becomes a Risk: Authenticity and Safety Risks in the Emerging Image Generation Paradigmcited by 2
  36. 2024Can Large Language Models Automatically Jailbreak GPT-4V?cited by 1
  37. 2025Mapping User Trust in Vision Language Models: Research Landscape, Challenges, and Prospectscited by 1
  38. 2025MDIT-Bench: Evaluating the Dual-Implicit Toxicity in Large Multimodal Modelscited by 1

50 most cited of 70 All 70 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024MLLMGuard: A Multi-dimensional Safety Evaluation Suite for Multimodal Large Language Models
  2. 2025SDEval: Safety Dynamic Evaluation for Multimodal Large Language Models
  3. 2025Do We Really Need Curated Malicious Data for Safety Alignment in Multi-modal Large Language Models?
  4. 2024CoCA: Regaining Safety-awareness of Multimodal Large Language Models with Constitutional Calibration
  5. 2025Pushing the Limits of Safety: A Technical Report on the ATLAS Challenge 2025
  6. 2025USB: A Comprehensive and Unified Safety Evaluation Benchmark for Multimodal Large Language Models
  7. 2024VLSBench: Unveiling Visual Leakage in Multimodal Safety
  8. 2026SafeSteer: A Decoding-level Defense Mechanism for Multimodal Large Language Models