mlsanityData to 14 Oct 2026

Seeing Through Deception: Uncovering Misleading Creator Intent in Multimodal News with Vision-Language Models

arXiv 2505.15489 ↗First posted 21 May 2025cs.CV cs.CL cs.MMcited by 5

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

Abstract (arXiv)

The impact of multimodal misinformation arises not only from factual inaccuracies but also from the misleading narratives that creators deliberately embed. Interpreting such creator intent is therefore essential for multimodal misinformation detection (MMD) and effective information governance. To this end, we introduce DeceptionDecoded, a large-scale benchmark of 12,000 image-caption pairs grounded in trustworthy reference articles, created using an intent-guided simulation framework that models both the desired influence and the execution plan of news creators. The dataset captures both misleading and non-misleading cases, spanning manipulations across visual and textual modalities, and supports three intent-centric tasks: (1) misleading intent detection, (2) misleading source attribution, and (3) creator desire inference. We evaluate 14 state-of-the-art vision-language models (VLMs) and find that they struggle with intent reasoning, often relying on shallow cues such as surface-level alignment, stylistic polish, or heuristic authenticity signals. To bridge this, our framework systematically synthesizes data that enables models to learn implication-level intent reasoning. Models trained on DeceptionDecoded demonstrate strong transferability to real-world MMD, validating our framework as both a benchmark to diagnose VLM fragility and a data synthesis engine that provides high-quality, intent-focused resources for enhancing robustness in real-world multimodal misinformation governance.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 31 of its 64 references that are in the map. First posted on arXiv 21 May 2025.

Builds on

31 of 64 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024SNIFFER: Multimodal Large Language Model for Explainable Out-of-Context Misinformation Detection5 Mar

Other references · 30

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2023Visual Instruction Tuningcited by 7,869
  4. 2025Qwen2.5-VL Technical Reportcited by 5,587
  5. 2024GPT-4o System Cardcited by 4,588
  6. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  7. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
  8. 2024A Survey on LLM-as-a-Judgecited by 1,423
  9. 2023Towards Understanding Sycophancy in Language Modelscited by 1,040
  10. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 573
  11. 2018FakeNewsNet: A Data Repository with News Content, Social Context and Spatialtemporal Information for Studying Fake News on Social Mediacited by 285
  12. 2023Can LLM-Generated Misinformation Be Detected?cited by 177
18 more
  1. 2020Visual News: Benchmark and Challenges in News Image Captioningcited by 121
  2. 2021NewsCLIPpings: Automatic Generation of Out-of-Context Multimodal Mediacited by 107
  3. 2022End-to-End Multimodal Fact-Checking and Explanation Generation: A Challenging Dataset and Modelscited by 107
  4. 2023Detecting and Grounding Multi-Modal Media Manipulationcited by 99
  5. 2025LLM Social Simulations Are a Promising Research Methodcited by 99
  6. 2019Transforming Delete, Retrieve, Generate Approach for Controlled Text Style Transfercited by 81
  7. 2019A Richly Annotated Corpus for Different Tasks in Automated Fact-Checkingcited by 74
  8. 2023Fake News in Sheep's Clothing: Robust Fake News Detection Against LLM-Empowered Style Attackscited by 70
  9. 2024MMFakeBench: A Mixed-Source Multimodal Misinformation Detection Benchmark for LVLMscited by 61
  10. 2021A Survey on Multimodal Disinformation Detectioncited by 47
  11. 2024DEFAME: Dynamic Evidence-based FAct-checking with Multimodal Expertscited by 43
  12. 2021Misinfo Reaction Frames: Reasoning about Readers' Reactions to News Headlinescited by 32
  13. 2024LEMMA: Towards LVLM-Enhanced Multimodal Misinformation Detection with External Knowledge Augmentationcited by 28
  14. 2023Primacy Effect of ChatGPTcited by 27
  15. 2023Support or Refute: Analyzing the Stance of Evidence to Detect Out-of-Context Mis- and Disinformationcited by 21
  16. 2020Edited Media Understanding Frames: Reasoning About the Intent and Implications of Visual Misinformationcited by 15
  17. 2024Customized large language models can outperform Community Notes in correcting misinformationcited by 9
  18. 2023Exploring news intent and its application: A theory-driven approachcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 4.

5 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025Towards Unified Multimodal Misinformation Detection in Social Media: A Benchmark Dataset and Baseline
  2. 2026Fact or Fake? Assessing the Role of Deepfake Detectors in Multimodal Misinformation Detection
  3. 2026Can Multimodal Large Language Models Generate and Detect Multimodal Social Media Fake News?
  4. 2025Debunk and Infer: Multimodal Fake News Detection via Diffusion-Generated Evidence and LLM Reasoning
  5. 2026The Synthetic Media Shift: Tracking the Rise, Virality, and Detectability of AI-Generated Multimodal Misinformation
  6. 2026DIVER: Dynamic Iterative Visual Evidence Reasoning for Multimodal Fake News Detection
  7. 2026The Paradigm Shift: A Comprehensive Survey on Large Vision Language Models for Multimodal Fake News Detection
  8. 2026MIC: Explaining Image-Claim Inconsistencies in AI-Generated Multimodal Misinformation