Seeing Through Deception: Uncovering Misleading Creator Intent in Multimodal News with Vision-Language Models
On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness
Abstract (arXiv)
The impact of multimodal misinformation arises not only from factual inaccuracies but also from the misleading narratives that creators deliberately embed. Interpreting such creator intent is therefore essential for multimodal misinformation detection (MMD) and effective information governance. To this end, we introduce DeceptionDecoded, a large-scale benchmark of 12,000 image-caption pairs grounded in trustworthy reference articles, created using an intent-guided simulation framework that models both the desired influence and the execution plan of news creators. The dataset captures both misleading and non-misleading cases, spanning manipulations across visual and textual modalities, and supports three intent-centric tasks: (1) misleading intent detection, (2) misleading source attribution, and (3) creator desire inference. We evaluate 14 state-of-the-art vision-language models (VLMs) and find that they struggle with intent reasoning, often relying on shallow cues such as surface-level alignment, stylistic polish, or heuristic authenticity signals. To bridge this, our framework systematically synthesizes data that enables models to learn implication-level intent reasoning. Models trained on DeceptionDecoded demonstrate strong transferability to real-world MMD, validating our framework as both a benchmark to diagnose VLM fragility and a data synthesis engine that provides high-quality, intent-focused resources for enhancing robustness in real-world multimodal misinformation governance.
Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 31 of its 64 references that are in the map. First posted on arXiv 21 May 2025.
Builds on
31 of 64 references.
Closest ancestors · 1
Same direction, newest first
- 2024SNIFFER: Multimodal Large Language Model for Explainable Out-of-Context Misinformation Detection5 Mar
Other references · 30
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2023Visual Instruction Tuningcited by 7,869
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2024GPT-4o System Cardcited by 4,588
- 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
- 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
- 2024A Survey on LLM-as-a-Judgecited by 1,423
- 2023Towards Understanding Sycophancy in Language Modelscited by 1,040
- 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 573
- 2018FakeNewsNet: A Data Repository with News Content, Social Context and Spatialtemporal Information for Studying Fake News on Social Mediacited by 285
- 2023Can LLM-Generated Misinformation Be Detected?cited by 177
18 more
- 2020Visual News: Benchmark and Challenges in News Image Captioningcited by 121
- 2021NewsCLIPpings: Automatic Generation of Out-of-Context Multimodal Mediacited by 107
- 2022End-to-End Multimodal Fact-Checking and Explanation Generation: A Challenging Dataset and Modelscited by 107
- 2023Detecting and Grounding Multi-Modal Media Manipulationcited by 99
- 2025LLM Social Simulations Are a Promising Research Methodcited by 99
- 2019Transforming Delete, Retrieve, Generate Approach for Controlled Text Style Transfercited by 81
- 2019A Richly Annotated Corpus for Different Tasks in Automated Fact-Checkingcited by 74
- 2023Fake News in Sheep's Clothing: Robust Fake News Detection Against LLM-Empowered Style Attackscited by 70
- 2024MMFakeBench: A Mixed-Source Multimodal Misinformation Detection Benchmark for LVLMscited by 61
- 2021A Survey on Multimodal Disinformation Detectioncited by 47
- 2024DEFAME: Dynamic Evidence-based FAct-checking with Multimodal Expertscited by 43
- 2021Misinfo Reaction Frames: Reasoning about Readers' Reactions to News Headlinescited by 32
- 2024LEMMA: Towards LVLM-Enhanced Multimodal Misinformation Detection with External Knowledge Augmentationcited by 28
- 2023Primacy Effect of ChatGPTcited by 27
- 2023Support or Refute: Analyzing the Stance of Evidence to Detect Out-of-Context Mis- and Disinformationcited by 21
- 2020Edited Media Understanding Frames: Reasoning About the Intent and Implications of Visual Misinformationcited by 15
- 2024Customized large language models can outperform Community Notes in correcting misinformationcited by 9
- 2023Exploring news intent and its application: A theory-driven approachcited by 7
Built on by
Citing papers per year: 2025: 1; 2026: 4.
5 papers.
- 2025M4FC: a Multimodal, Multilingual, Multicultural, Multitask Real-World Fact-Checking Datasetcited by 7
- 2026What's Left Unsaid? Detecting and Correcting Misleading Omissions in Multimodal News Previewscited by 3
- 2026Correct When Paired, Wrong When Split: Decoupling and Editing Modality-Specific Neurons in MLLMscited by 1
- 2026Multimodal Climate Disinformation Detection: Integrating Vision-Language Models with External Knowledge Sourcescited by 0
- 2026When Visuals Aren't the Problem: Evaluating Vision-Language Models on Misleading Data Visualizationscited by 0
- 2026Correct When Paired, Wrong When Split: Decoupling and Editing Modality-Specific Neurons in MLLMs20 Apr 2026
- 2026When Visuals Aren't the Problem: Evaluating Vision-Language Models on Misleading Data Visualizations23 Mar 2026
- 2026Multimodal Climate Disinformation Detection: Integrating Vision-Language Models with External Knowledge Sources22 Jan 2026
- 2026What's Left Unsaid? Detecting and Correcting Misleading Omissions in Multimodal News Previews9 Jan 2026
- 2025M4FC: a Multimodal, Multilingual, Multicultural, Multitask Real-World Fact-Checking Dataset27 Oct 2025
Similar, not cited
Close in wording, no citation either way
- 2025Towards Unified Multimodal Misinformation Detection in Social Media: A Benchmark Dataset and Baseline
- 2026Fact or Fake? Assessing the Role of Deepfake Detectors in Multimodal Misinformation Detection
- 2026Can Multimodal Large Language Models Generate and Detect Multimodal Social Media Fake News?
- 2025Debunk and Infer: Multimodal Fake News Detection via Diffusion-Generated Evidence and LLM Reasoning
- 2026The Synthetic Media Shift: Tracking the Rise, Virality, and Detectability of AI-Generated Multimodal Misinformation
- 2026DIVER: Dynamic Iterative Visual Evidence Reasoning for Multimodal Fake News Detection
- 2026The Paradigm Shift: A Comprehensive Survey on Large Vision Language Models for Multimodal Fake News Detection
- 2026MIC: Explaining Image-Claim Inconsistencies in AI-Generated Multimodal Misinformation