# Synthetic Misinformers: Generating and Combating Multimodal Misinformation

arXiv: https://arxiv.org/abs/2303.01217 · HTML: https://mlsanity.com/paper/2303.01217/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 31 references that are in the map.
- First posted on arXiv 2 Mar 2023.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Robust fine-tuning of zero-shot models](https://mlsanity.com/paper/2109.01903/) (2021)
- [RUBi: Reducing Unimodal Biases in Visual Question Answering](https://mlsanity.com/paper/1906.10169/) (2019)
- [Visual News: Benchmark and Challenges in News Image Captioning](https://mlsanity.com/paper/2010.03743/) (2020)
- [NewsCLIPpings: Automatic Generation of Out-of-Context Multimodal Media](https://mlsanity.com/paper/2104.05893/) (2021)
- [Open-Domain, Content-based, Multi-modal Fact-checking of Out-of-Context Images via Online Resources](https://mlsanity.com/paper/2112.00061/) (2021)
- [A Survey on Multimodal Disinformation Detection](https://mlsanity.com/paper/2103.12541/) (2021)
- [Missing Counter-Evidence Renders NLP Fact-Checking Unrealistic for Misinformation](https://mlsanity.com/paper/2210.13865/) (2022)
- [Multimodal Analytics for Real-world News using Measures of Cross-modal Entity Consistency](https://mlsanity.com/paper/2003.10421/) (2020)
- [Deep Multimodal Image-Repurposing Detection](https://mlsanity.com/paper/1808.06686/) (2018)
- [Multimedia Semantic Integrity Assessment Using Joint Embedding Of Images And Text](https://mlsanity.com/paper/1707.01606/) (2017)
- [COSMOS: Catching Out-of-Context Misinformation with Self-Supervised Learning](https://mlsanity.com/paper/2101.06278/) (2021)
- [Twitter-COMMs: Detecting Climate, COVID, and Military Multimodal Misinformation](https://mlsanity.com/paper/2112.08594/) (2021)
- [MMSys'21 Grand Challenge on Detecting Cheapfakes](https://mlsanity.com/paper/2107.05297/) (2021)

## Built on by (most cited first, 23 of 23)

- [Sociotechnical Safety Evaluation of Generative AI Systems](https://mlsanity.com/paper/2310.11986/) (2023)
- [SNIFFER: Multimodal Large Language Model for Explainable Out-of-Context Misinformation Detection](https://mlsanity.com/paper/2403.03170/) (2024)
- [VERITE: A Robust Benchmark for Multimodal Misinformation Detection Accounting for Unimodal Bias](https://mlsanity.com/paper/2304.14133/) (2023)
- [Support or Refute: Analyzing the Stance of Evidence to Detect Out-of-Context Mis- and Disinformation](https://mlsanity.com/paper/2311.01766/) (2023)
- [RED-DOT: Multimodal Fact-checking via Relevant Evidence Detection](https://mlsanity.com/paper/2311.09939/) (2023)
- [Similarity over Factuality: Are we making progress on multimodal out-of-context misinformation detection?](https://mlsanity.com/paper/2407.13488/) (2024)
- [VLDBench Evaluating Multimodal Disinformation with Regulatory Alignment](https://mlsanity.com/paper/2502.11361/) (2025)
- [Interpretable Detection of Out-of-Context Misinformation with Neural-Symbolic-Enhanced Large Multimodal Model](https://mlsanity.com/paper/2304.07633/) (2023)
- [CMIE: Combining MLLM Insights with External Evidence for Explainable Out-of-Context Misinformation Detection](https://mlsanity.com/paper/2505.23449/) (2025)
- [The Next Phase of Scientific Fact-Checking: Advanced Evidence Retrieval from Complex Structured Academic Papers](https://mlsanity.com/paper/2506.20844/) (2025)
- [Verifying Cross-modal Entity Consistency in News using Vision-language Models](https://mlsanity.com/paper/2501.11403/) (2025)
- ["Image, Tell me your story!" Predicting the original meta-context of visual misinformation](https://mlsanity.com/paper/2408.09939/) (2024)
- [MAD-Sherlock: Multi-Agent Debate for Visual Misinformation Detection](https://mlsanity.com/paper/2410.20140/) (2024)
- [HintsOfTruth: A Multimodal Checkworthiness Detection Dataset with Real and Synthetic Claims](https://mlsanity.com/paper/2502.11753/) (2025)
- [EXCLAIM: An Explainable Cross-Modal Agentic System for Misinformation Detection with Hierarchical Retrieval](https://mlsanity.com/paper/2504.06269/) (2025)
- [Multi-MLLM Knowledge Distillation for Out-of-Context News Detection](https://mlsanity.com/paper/2505.22517/) (2025)
- ["Humor, Art, or Misinformation?": A Multimodal Dataset for Intent-Aware Synthetic Image Detection](https://mlsanity.com/paper/2508.20670/) (2025)
- [RW-Post: Auditable Evidence-Grounded Multimodal Fact-Checking in the Wild](https://mlsanity.com/paper/2605.10357/) (2026)
- [Can Out-of-Domain data help to Learn Domain-Specific Prompts for Multimodal Misinformation Detection?](https://mlsanity.com/paper/2311.16496/) (2023)
- [Mitigating GenAI-Powered Evidence Pollution for Out-Of-Context Misinformation Detection](https://mlsanity.com/paper/2501.14728/) (2025)
- [Latent Reconstruction from Generated Data for Multimodal Misinformation Detection](https://mlsanity.com/paper/2504.06010/) (2025)
- [RW-Post: Auditable Evidence-Grounded Multimodal Fact-Checking in the Wild](https://mlsanity.com/paper/2512.22933/) (2025)
- [Evidence Triangulation for Multimodal Fact-Checking in the Wild](https://mlsanity.com/paper/2606.31367/) (2026)

## Abstract (arXiv)

With the expansion of social media and the increasing dissemination of multimedia content, the spread of misinformation has become a major concern. This necessitates effective strategies for multimodal misinformation detection (MMD) that detect whether the combination of an image and its accompanying text could mislead or misinform. Due to the data-intensive nature of deep neural networks and the labor-intensive process of manual annotation, researchers have been exploring various methods for automatically generating synthetic multimodal misinformation - which we refer to as Synthetic Misinformers - in order to train MMD models. However, limited evaluation on real-world misinformation and a lack of comparisons with other Synthetic Misinformers makes difficult to assess progress in the field. To address this, we perform a comparative study on existing and new Synthetic Misinformers that involves (1) out-of-context (OOC) image-caption pairs, (2) cross-modal named entity inconsistency (NEI) as well as (3) hybrid approaches and we evaluate them against real-world misinformation; using the COSMOS benchmark. The comparative study showed that our proposed CLIP-based Named Entity Swapping can lead to MMD models that surpass other OOC and NEI Misinformers in terms of multimodal accuracy and that hybrid approaches can lead to even higher detection accuracy. Nevertheless, after alleviating information leakage from the COSMOS evaluation protocol, low Sensitivity scores indicate that the task is significantly more challenging than previous studies suggested. Finally, our findings showed that NEI-based Synthetic Misinformers tend to suffer from a unimodal bias, where text-only MMDs can outperform multimodal ones.