# MLLMGuard: A Multi-dimensional Safety Evaluation Suite for Multimodal Large Language Models

arXiv: https://arxiv.org/abs/2406.07594 · HTML: https://mlsanity.com/paper/2406.07594/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 92 of its 134 references that are in the map.
- First posted on arXiv 11 Jun 2024.

## Placed in

- Multimodal LLMs / vision-language models / [MLLM jailbreaks and adversarial robustness](https://mlsanity.com/mllm/mllm-jailbreaks-and-adversarial-robustness/)

## Builds on (most cited first, 30 of 92)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models](https://mlsanity.com/paper/2304.10592/) (2023)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023)
- [Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering](https://mlsanity.com/paper/1612.00837/) (2016)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering](https://mlsanity.com/paper/2209.09513/) (2022)
- [Towards VQA Models That Can Read](https://mlsanity.com/paper/1904.08920/) (2019)
- [MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models](https://mlsanity.com/paper/2306.13394/) (2023)
- [LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs](https://mlsanity.com/paper/2111.02114/) (2021)
- [ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning](https://mlsanity.com/paper/2203.10244/) (2022)
- [Microsoft COCO Captions: Data Collection and Evaluation Server](https://mlsanity.com/paper/1504.00325/) (2015)
- [Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models](https://mlsanity.com/paper/1505.04870/) (2015)
- [DocVQA: A Dataset for VQA on Document Images](https://mlsanity.com/paper/2007.00398/) (2020)
- [Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts](https://mlsanity.com/paper/2102.08981/) (2021)
- [OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledge](https://mlsanity.com/paper/1906.00067/) (2019)
- [Kosmos-2: Grounding Multimodal Large Language Models to the World](https://mlsanity.com/paper/2306.14824/) (2023)
- [Modeling Context in Referring Expressions](https://mlsanity.com/paper/1608.00272/) (2016)
- [How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites](https://mlsanity.com/paper/2404.16821/) (2024)
- [SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehension](https://mlsanity.com/paper/2307.16125/) (2023)
- [Are We on the Right Way for Evaluating Large Vision-Language Models?](https://mlsanity.com/paper/2403.20330/) (2024)

## Built on by (most cited first, 30 of 34)

- [VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Models](https://mlsanity.com/paper/2407.11691/) (2024)
- [Qwen3Guard Technical Report](https://mlsanity.com/paper/2510.14276/) (2025)
- [A Survey of Safety on Large Vision-Language Models: Attacks, Defenses and Evaluations](https://mlsanity.com/paper/2502.14881/) (2025)
- [Hume: Introducing System-2 Thinking in Visual-Language-Action Model](https://mlsanity.com/paper/2505.21432/) (2025)
- [GuardReasoner-VL: Safeguarding VLMs via Reinforced Reasoning](https://mlsanity.com/paper/2505.11049/) (2025)
- [Safe RLHF-V: Safe Reinforcement Learning from Multi-modal Human Feedback](https://mlsanity.com/paper/2503.17682/) (2025)
- [Video-SafetyBench: A Benchmark for Safety Evaluation of Video LVLMs](https://mlsanity.com/paper/2505.11842/) (2025)
- [A Survey on Multimodal Benchmarks: In the Era of Large AI Models](https://mlsanity.com/paper/2409.18142/) (2024)
- [A Survey on Agentic Multimodal Large Language Models](https://mlsanity.com/paper/2510.10991/) (2025)
- [VLMGuard: Bootstrapping Malicious Prompt Detectors from Unlabeled Vision-Language Prompts in the Wild](https://mlsanity.com/paper/2410.00296/) (2024)
- [Safety Mirage: How Spurious Correlations Undermine VLM Safety Fine-Tuning and Can Be Mitigated by Machine Unlearning](https://mlsanity.com/paper/2503.11832/) (2025)
- [JailDAM: Jailbreak Detection with Adaptive Memory for Vision-Language Model](https://mlsanity.com/paper/2504.03770/) (2025)
- [USB: A Comprehensive and Unified Safety Evaluation Benchmark for Multimodal Large Language Models](https://mlsanity.com/paper/2505.23793/) (2025)
- [SaFeR-VLM: Toward Safety-aware Fine-grained Reasoning in Multimodal Models](https://mlsanity.com/paper/2510.06871/) (2025)
- [VLSU: Mapping the Limits of Joint Multimodal Understanding for AI Safety](https://mlsanity.com/paper/2510.18214/) (2025)
- [ELITE: Enhanced Language-Image Toxicity Evaluation for Safety](https://mlsanity.com/paper/2502.04757/) (2025)
- [REVEAL: Multi-turn Evaluation of Image-Input Harms for Vision LLM](https://mlsanity.com/paper/2505.04673/) (2025)
- [Multi-PA: A Multi-perspective Benchmark on Privacy Assessment for Large Vision-Language Models](https://mlsanity.com/paper/2412.19496/) (2024)
- [MIST: Jailbreaking Black-box Large Language Models via Iterative Semantic Tuning](https://mlsanity.com/paper/2506.16792/) (2025)
- [OutSafe-Bench: A Benchmark for Multimodal Offensive Content Detection in Large Language Models](https://mlsanity.com/paper/2511.10287/) (2025)
- [Who Can See Through You? Adversarial Shielding Against VLM-Based Attribute Inference Attacks](https://mlsanity.com/paper/2512.18264/) (2025)
- [On the Adversarial Robustness of Multimodal LLM Judges](https://mlsanity.com/paper/2606.15608/) (2026)
- [From Evaluation to Defense: Advancing Safety in Video Large Language Models](https://mlsanity.com/paper/2505.16643/) (2025)
- [From Individuals to Interactions: Benchmarking Gender Bias in Multimodal Large Language Models from the Lens of Social Relationship](https://mlsanity.com/paper/2506.23101/) (2025)
- [Lingua-SafetyBench: A Benchmark for Safety Evaluation of Multilingual Vision-Language Models](https://mlsanity.com/paper/2601.22737/) (2026)
- [When Understanding Becomes a Risk: Authenticity and Safety Risks in the Emerging Image Generation Paradigm](https://mlsanity.com/paper/2603.24079/) (2026)
- [Multimodal Reward Hacking in Reinforcement Learning](https://mlsanity.com/paper/2607.09492/) (2026)
- [Argus Inspection: Do Multimodal Large Language Models Possess the Eye of Panoptes?](https://mlsanity.com/paper/2506.14805/) (2025)
- [LLaVAShield: Safeguarding Multimodal Multi-Turn Dialogues in Vision-Language Models](https://mlsanity.com/paper/2509.25896/) (2025)
- [From Sparse Decisions to Dense Reasoning: A Multi-attribute Trajectory Paradigm for Multimodal Moderation](https://mlsanity.com/paper/2602.02536/) (2026)

## Abstract (arXiv)

Powered by remarkable advancements in Large Language Models (LLMs), Multimodal Large Language Models (MLLMs) demonstrate impressive capabilities in manifold tasks. However, the practical application scenarios of MLLMs are intricate, exposing them to potential malicious instructions and thereby posing safety risks. While current benchmarks do incorporate certain safety considerations, they often lack comprehensive coverage and fail to exhibit the necessary rigor and robustness. For instance, the common practice of employing GPT-4V as both the evaluator and a model to be evaluated lacks credibility, as it tends to exhibit a bias toward its own responses. In this paper, we present MLLMGuard, a multidimensional safety evaluation suite for MLLMs, including a bilingual image-text evaluation dataset, inference utilities, and a lightweight evaluator. MLLMGuard's assessment comprehensively covers two languages (English and Chinese) and five important safety dimensions (Privacy, Bias, Toxicity, Truthfulness, and Legality), each with corresponding rich subtasks. Focusing on these dimensions, our evaluation dataset is primarily sourced from platforms such as social media, and it integrates text-based and image-based red teaming techniques with meticulous annotation by human experts. This can prevent inaccurate evaluation caused by data leakage when using open-source datasets and ensures the quality and challenging nature of our benchmark. Additionally, a fully automated lightweight evaluator termed GuardRank is developed, which achieves significantly higher evaluation accuracy than GPT-4. Our evaluation results across 13 advanced models indicate that MLLMs still have a substantial journey ahead before they can be considered safe and responsible.