# VLDBench Evaluating Multimodal Disinformation with Regulatory Alignment

arXiv: https://arxiv.org/abs/2502.11361 · HTML: https://mlsanity.com/paper/2502.11361/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 46 of its 128 references that are in the map.
- First posted on arXiv 17 Feb 2025.

## Builds on (most cited first, 30 of 46)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023)
- [Qwen2 Technical Report](https://mlsanity.com/paper/2407.10671/) (2024)
- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model](https://mlsanity.com/paper/2405.04434/) (2024)
- [ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools](https://mlsanity.com/paper/2406.12793/) (2024)
- [FaceForensics++: Learning to Detect Manipulated Facial Images](https://mlsanity.com/paper/1901.08971/) (2019)
- [ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks](https://mlsanity.com/paper/2303.15056/) (2023)
- [HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Models](https://mlsanity.com/paper/2310.14566/) (2023)
- [Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensics](https://mlsanity.com/paper/1909.12962/) (2019)
- [DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding](https://mlsanity.com/paper/2412.10302/) (2024)
- [Defending Against Neural Fake News](https://mlsanity.com/paper/1905.12616/) (2019)
- ["Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detection](https://mlsanity.com/paper/1705.00648/) (2017)
- [Fake News Detection on Social Media: A Data Mining Perspective](https://mlsanity.com/paper/1708.01967/) (2017)
- [FakeNewsNet: A Data Repository with News Content, Social Context and Spatialtemporal Information for Studying Fake News on Social Media](https://mlsanity.com/paper/1809.01286/) (2018)
- [MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models](https://mlsanity.com/paper/2311.17600/) (2023)
- [Rainbow Teaming: Open-Ended Generation of Diverse Adversarial Prompts](https://mlsanity.com/paper/2402.16822/) (2024)
- [SafetyBench: Evaluating the Safety of Large Language Models](https://mlsanity.com/paper/2309.07045/) (2023)
- [Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Models](https://mlsanity.com/paper/2402.02207/) (2024)
- [NewsCLIPpings: Automatic Generation of Out-of-Context Multimodal Media](https://mlsanity.com/paper/2104.05893/) (2021)
- [Combating Misinformation in the Age of LLMs: Opportunities and Challenges](https://mlsanity.com/paper/2311.05656/) (2023)
- [SNIFFER: Multimodal Large Language Model for Explainable Out-of-Context Misinformation Detection](https://mlsanity.com/paper/2403.03170/) (2024)
- [Introducing v0.5 of the AI Safety Benchmark from MLCommons](https://mlsanity.com/paper/2404.12241/) (2024)
- [MuMiN: A Large-Scale Multilingual Multimodal Fact-Checked Misinformation Social Network Dataset](https://mlsanity.com/paper/2202.11684/) (2022)
- [Detecting and Grounding Multi-Modal Media Manipulation and Beyond](https://mlsanity.com/paper/2309.14203/) (2023)
- [Examining Gender and Racial Bias in Large Vision-Language Models Using a Novel Dataset of Parallel Images](https://mlsanity.com/paper/2402.05779/) (2024)

## Built on by (most cited first, 9 of 9)

- [Thinking Beyond Tokens: From Brain-Inspired Intelligence to Cognitive Foundations for Artificial General Intelligence and its Societal Impact](https://mlsanity.com/paper/2507.00951/) (2025)
- [A Review of 3D Object Detection with Vision-Language Models](https://mlsanity.com/paper/2504.18738/) (2025)
- [Compose and Fuse: Revisiting the Foundational Bottlenecks in Multimodal Reasoning](https://mlsanity.com/paper/2509.23744/) (2025)
- [All That Glisters Is Not Gold: A Benchmark for Reference-Free Counterfactual Financial Misinformation Detection](https://mlsanity.com/paper/2601.04160/) (2026)
- [Reducing Hallucinations in LLMs via Factuality-Aware Preference Learning](https://mlsanity.com/paper/2601.03027/) (2026)
- [FairSense-AI: Responsible AI Meets Sustainability](https://mlsanity.com/paper/2503.02865/) (2025)
- [COREVQA: A Crowd Observation and Reasoning Entailment Visual Question Answering Benchmark](https://mlsanity.com/paper/2507.13405/) (2025)
- [What Improves Multimodal Misinformation Detection? Answers from a Large-Scale Empirical Study](https://mlsanity.com/paper/2609.30402/) (2026)
- [Geometric Parrots: Rule Absence vs. Rule Fragility in Spatial and Physical Reasoning for Vision-Language Models](https://mlsanity.com/paper/2610.16170/) (2026)

## Abstract (arXiv)

Detecting disinformation that blends manipulated text and images has become increasingly challenging, as AI tools make synthetic content easy to generate and disseminate. While most existing AI safety benchmarks focus on single modality misinformation (i.e., false content shared without intent to deceive), intentional multimodal disinformation, such as propaganda or conspiracy theories that imitate credible news, remains largely unaddressed. We introduce the Vision-Language Disinformation Detection Benchmark (VLDBench), the first large-scale resource supporting both unimodal (text-only) and multimodal (text + image) disinformation detection. VLDBench comprises approximately 62,000 labeled text-image pairs across 13 categories, curated from 58 news outlets. Using a semi-automated pipeline followed by expert review, 22 domain experts invested over 500 hours to produce high-quality annotations with substantial inter-annotator agreement. Evaluations of state-of-the-art Large Language Models (LLMs) and Vision-Language Models (VLMs) on VLDBench show that incorporating visual cues improves detection accuracy by 5 to 35 percentage points over text-only models. VLDBench provides data and code for evaluation, fine-tuning, and robustness testing to support disinformation analysis. Developed in alignment with AI governance frameworks (e.g., the MIT AI Risk Repository), VLDBench offers a principled foundation for advancing trustworthy disinformation detection in multimodal media. Project: https://vectorinstitute.github.io/VLDBench/ Dataset: https://huggingface.co/datasets/vector-institute/VLDBench Code: https://github.com/VectorInstitute/VLDBench