# Multimodal Misinformation Detection by Learning from Synthetic Data with Multimodal LLMs

arXiv: https://arxiv.org/abs/2409.19656 · HTML: https://mlsanity.com/paper/2409.19656/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 48 of its 81 references that are in the map.
- First posted on arXiv 29 Sep 2024.

## Placed in

- Multimodal LLMs / vision-language models / [MLLM jailbreaks and adversarial robustness](https://mlsanity.com/mllm/mllm-jailbreaks-and-adversarial-robustness/)
- Synthetic data and self-improvement for LLMs / [LLM synthetic data generation](https://mlsanity.com/synth/llm-synthetic-data-generation/)

## Builds on (most cited first, 30 of 48)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery](https://mlsanity.com/paper/2103.17249/) (2021)
- [Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs](https://mlsanity.com/paper/2401.06209/) (2024)
- [DataComp: In search of the next generation of multimodal datasets](https://mlsanity.com/paper/2304.14108/) (2023)
- [Data Shapley: Equitable Valuation of Data for Machine Learning](https://mlsanity.com/paper/1904.02868/) (2019)
- [Detecting Twenty-thousand Classes using Image-level Supervision](https://mlsanity.com/paper/2201.02605/) (2022)
- [mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaboration](https://mlsanity.com/paper/2311.04257/) (2023)
- [LESS: Selecting Influential Data for Targeted Instruction Tuning](https://mlsanity.com/paper/2402.04333/) (2024)
- [PandaGPT: One Model To Instruction-Follow Them All](https://mlsanity.com/paper/2305.16355/) (2023)
- [Data Selection for Language Models via Importance Resampling](https://mlsanity.com/paper/2302.03169/) (2023)
- [TRAK: Attributing Model Behavior at Scale](https://mlsanity.com/paper/2303.14186/) (2023)
- [Joint Distribution Optimal Transportation for Domain Adaptation](https://mlsanity.com/paper/1705.08848/) (2017)
- [SimSwap: An Efficient Framework For High Fidelity Face Swapping](https://mlsanity.com/paper/2106.06340/) (2021)
- [A Survey on Data Selection for Language Models](https://mlsanity.com/paper/2402.16827/) (2024)
- [High-Fidelity GAN Inversion for Image Attribute Editing](https://mlsanity.com/paper/2109.06590/) (2021)
- [Fighting Fake News: Image Splice Detection via Learned Self-Consistency](https://mlsanity.com/paper/1805.04096/) (2018)
- [Visual News: Benchmark and Challenges in News Image Captioning](https://mlsanity.com/paper/2010.03743/) (2020)
- [NewsCLIPpings: Automatic Generation of Out-of-Context Multimodal Media](https://mlsanity.com/paper/2104.05893/) (2021)
- [End-to-End Multimodal Fact-Checking and Explanation Generation: A Challenging Dataset and Models](https://mlsanity.com/paper/2205.12487/) (2022)
- [Detecting and Grounding Multi-Modal Media Manipulation](https://mlsanity.com/paper/2304.02556/) (2023)
- [SNIFFER: Multimodal Large Language Model for Explainable Out-of-Context Misinformation Detection](https://mlsanity.com/paper/2403.03170/) (2024)
- [Open-Domain, Content-based, Multi-modal Fact-checking of Out-of-Context Images via Online Resources](https://mlsanity.com/paper/2112.00061/) (2021)
- [LAVA: Data Valuation without Pre-Specified Learning Algorithms](https://mlsanity.com/paper/2305.00054/) (2023)
- [SBERT-WK: A Sentence Embedding Method by Dissecting BERT-based Word Models](https://mlsanity.com/paper/2002.06652/) (2020)

## Built on by (most cited first, 13 of 13)

- [DEFAME: Dynamic Evidence-based FAct-checking with Multimodal Experts](https://mlsanity.com/paper/2412.10510/) (2024)
- [AVerImaTeC: A Dataset for Automatic Verification of Image-Text Claims with Evidence from the Web](https://mlsanity.com/paper/2505.17978/) (2025)
- [XFacta: Contemporary, Real-World Dataset and Evaluation for Multimodal Misinformation Detection with Multimodal LLMs](https://mlsanity.com/paper/2508.09999/) (2025)
- [Generating Attribution Reports for Manipulated Facial Images: A Dataset and Baseline](https://mlsanity.com/paper/2412.19685/) (2024)
- [The Paradigm Shift: A Comprehensive Survey on Large Vision Language Models for Multimodal Fake News Detection](https://mlsanity.com/paper/2601.15316/) (2026)
- [T^2Agent A Tool-augmented Multimodal Misinformation Detection Agent with Monte Carlo Tree Search](https://mlsanity.com/paper/2505.19768/) (2025)
- [E-CaTCH: Event-Centric Cross-Modal Attention with Temporal Consistency and Class-Imbalance Handling for Misinformation Detection](https://mlsanity.com/paper/2508.11197/) (2025)
- [Enhancing Multimodal Misinformation Detection by Replaying the Whole Story from Image Modality Perspective](https://mlsanity.com/paper/2511.06284/) (2025)
- [Multimodal Claim Extraction for Fact-Checking](https://mlsanity.com/paper/2604.16311/) (2026)
- [REVEAL: Reference-Grounded Reasoning for Multimodal Manipulation Detection](https://mlsanity.com/paper/2605.28459/) (2026)
- [SynCred-Bench: Benchmarking Synthetic Credibility in AI-Generated Visual Misinformation](https://mlsanity.com/paper/2606.03348/) (2026)
- [From Veracity to Diffusion: Adressing Operational Challenges in Moving From Fake-News Detection to Information Disorders](https://mlsanity.com/paper/2512.02552/) (2025)
- [Latent Fact-Checking: Detecting Misinformation through Activation Engineering](https://mlsanity.com/paper/2608.06417/) (2026)

## Abstract (arXiv)

Detecting multimodal misinformation, especially in the form of image-text pairs, is crucial. Obtaining large-scale, high-quality real-world fact-checking datasets for training detectors is costly, leading researchers to use synthetic datasets generated by AI technologies. However, the generalizability of detectors trained on synthetic data to real-world scenarios remains unclear due to the distribution gap. To address this, we propose learning from synthetic data for detecting real-world multimodal misinformation through two model-agnostic data selection methods that match synthetic and real-world data distributions. Experiments show that our method enhances the performance of a small MLLM (13B) on real-world fact-checking datasets, enabling it to even surpass GPT-4V~citeGPT-4V.