# Multi-modal Fake News Detection on Social Media via Multi-grained Information Fusion

arXiv: https://arxiv.org/abs/2304.00827 · HTML: https://mlsanity.com/paper/2304.00827/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 50 references that are in the map.
- First posted on arXiv 3 Apr 2023.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks](https://mlsanity.com/paper/1908.02265/) (2019)
- [Multimodal Machine Learning: A Survey and Taxonomy](https://mlsanity.com/paper/1705.09406/) (2017)
- [Open-vocabulary Object Detection via Vision and Language Knowledge Distillation](https://mlsanity.com/paper/2104.13921/) (2021)
- [Language-driven Semantic Segmentation](https://mlsanity.com/paper/2201.03546/) (2022)
- [FakeNewsNet: A Data Repository with News Content, Social Context and Spatialtemporal Information for Studying Fake News on Social Media](https://mlsanity.com/paper/1809.01286/) (2018)
- [Rumor Detection on Social Media with Bi-Directional Graph Convolutional Networks](https://mlsanity.com/paper/2001.06362/) (2020)
- [Detection and Resolution of Rumours in Social Media: A Survey](https://mlsanity.com/paper/1704.00656/) (2017)
- [HairCLIP: Design Your Hair by Text and Reference Image](https://mlsanity.com/paper/2112.05142/) (2021)
- [Beyond Question-Based Biases: Assessing Multimodal Shortcut Learning in Visual Question Answering](https://mlsanity.com/paper/2104.03149/) (2021)
- [Open-Domain, Content-based, Multi-modal Fact-checking of Out-of-Context Images via Online Resources](https://mlsanity.com/paper/2112.00061/) (2021)
- [MDFEND: Multi-domain Fake News Detection](https://mlsanity.com/paper/2201.00987/) (2022)
- [CLIP-Art: Contrastive Pre-training for Fine-Grained Art Classification](https://mlsanity.com/paper/2204.14244/) (2022)
- [Exploiting Multi-domain Visual Information for Fake News Detection](https://mlsanity.com/paper/1908.04472/) (2019)
- [Improving Fake News Detection by Using an Entity-enhanced Framework to Fuse Diverse Multimodal Clues](https://mlsanity.com/paper/2108.10509/) (2021)
- [Multimodal Fake News Detection via CLIP-Guided Learning](https://mlsanity.com/paper/2205.14304/) (2022)
- [Multimodal Analytics for Real-world News using Measures of Cross-modal Entity Consistency](https://mlsanity.com/paper/2003.10421/) (2020)
- [Memory-Guided Multi-View Multi-Domain Fake News Detection](https://mlsanity.com/paper/2206.12808/) (2022)
- [Bootstrapping Multi-view Representations for Fake News Detection](https://mlsanity.com/paper/2206.05741/) (2022)

## Built on by (most cited first, 15 of 15)

- [Unified Frequency-Assisted Transformer Framework for Detecting and Grounding Multi-Modal Manipulation](https://mlsanity.com/paper/2309.09667/) (2023)
- [Multimodal Misinformation Detection by Learning from Synthetic Data with Multimodal LLMs](https://mlsanity.com/paper/2409.19656/) (2024)
- [Modality Interactive Mixture-of-Experts for Fake News Detection](https://mlsanity.com/paper/2501.12431/) (2025)
- [Fake News Detection and Manipulation Reasoning via Large Vision-Language Models](https://mlsanity.com/paper/2407.02042/) (2024)
- [Unleashing the Potential of Consistency Learning for Detecting and Grounding Multi-Modal Media Manipulation](https://mlsanity.com/paper/2506.05890/) (2025)
- [KEN: Knowledge Augmentation and Emotion Guidance Network for Multimodal Fake News Detection](https://mlsanity.com/paper/2507.09647/) (2025)
- [Search, Examine and Early-Termination: Fake News Detection with Annotation-Free Evidences](https://mlsanity.com/paper/2407.07931/) (2024)
- [SEER: Semantic Enhancement and Emotional Reasoning Network for Multimodal Fake News Detection](https://mlsanity.com/paper/2507.13415/) (2025)
- [Multimodal Fake News Video Explanation: Dataset, Analysis and Evaluation](https://mlsanity.com/paper/2501.08514/) (2025)
- [Beyond Retrieval: Improving Evidence Quality for LLM-based Multimodal Fact-Checking](https://mlsanity.com/paper/2505.03135/) (2025)
- [Fact-checking based fake news detection: a review](https://mlsanity.com/paper/2401.01717/) (2024)
- [Adaptive Learning of Consistency and Inconsistency Information for Fake News Detection](https://mlsanity.com/paper/2408.08013/) (2024)
- [T3SVFND: Towards an Evolving Fake News Detector for Emergencies with Test-time Training on Short Video Platforms](https://mlsanity.com/paper/2507.20286/) (2025)
- [MM-FusionNet: Context-Aware Dynamic Fusion for Multi-modal Fake News Detection with Large Vision-Language Models](https://mlsanity.com/paper/2508.05684/) (2025)
- [Where Detectors Fail: Closing the Tail-Domain Gap with Expert-Guided Mutual Distillation](https://mlsanity.com/paper/2607.26555/) (2026)

## Abstract (arXiv)

The easy sharing of multimedia content on social media has caused a rapid dissemination of fake news, which threatens society's stability and security. Therefore, fake news detection has garnered extensive research interest in the field of social forensics. Current methods primarily concentrate on the integration of textual and visual features but fail to effectively exploit multi-modal information at both fine-grained and coarse-grained levels. Furthermore, they suffer from an ambiguity problem due to a lack of correlation between modalities or a contradiction between the decisions made by each modality. To overcome these challenges, we present a Multi-grained Multi-modal Fusion Network (MMFN) for fake news detection. Inspired by the multi-grained process of human assessment of news authenticity, we respectively employ two Transformer-based pre-trained models to encode token-level features from text and images. The multi-modal module fuses fine-grained features, taking into account coarse-grained features encoded by the CLIP encoder. To address the ambiguity problem, we design uni-modal branches with similarity-based weighting to adaptively adjust the use of multi-modal features. Experimental results demonstrate that the proposed framework outperforms state-of-the-art methods on three prevalent datasets.