# Multimodal rumor detection enhanced by external evidence and forgery features

arXiv: https://arxiv.org/abs/2601.14954 · HTML: https://mlsanity.com/paper/2601.14954/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 34 references that are in the map.
- First posted on arXiv 21 Jan 2026.

## Builds on (most cited first, 11 of 11)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Hierarchical Fine-Grained Image Forgery Detection and Localization](https://mlsanity.com/paper/2303.17111/) (2023)
- [Towards LLM-based Fact Verification on News Claims with a Hierarchical Step-by-Step Prompting Method](https://mlsanity.com/paper/2310.00305/) (2023)
- [Open-Domain, Content-based, Multi-modal Fact-checking of Out-of-Context Images via Online Resources](https://mlsanity.com/paper/2112.00061/) (2021)
- [Exploiting Multi-domain Visual Information for Fake News Detection](https://mlsanity.com/paper/1908.04472/) (2019)
- [Call Attention to Rumors: Deep Attention Based Recurrent Neural Networks for Early Rumor Detection](https://mlsanity.com/paper/1704.05973/) (2017)
- [Cross-modal Contrastive Learning for Multimodal Fake News Detection](https://mlsanity.com/paper/2302.14057/) (2023)
- [FakeGPT: Fake News Generation, Explanation and Detection of Large Language Models](https://mlsanity.com/paper/2310.05046/) (2023)
- [Multimodal Detection of Information Disorder from Social Media](https://mlsanity.com/paper/2105.15165/) (2021)
- [Cross-modal Consistency Learning with Fine-grained Fusion Network for Multimodal Fake News Detection](https://mlsanity.com/paper/2311.01807/) (2023)

## Abstract (arXiv)

Social media increasingly disseminates information through mixed image text posts, but rumors often exploit subtle inconsistencies and forged content, making detection based solely on post content difficult. Deep semantic mismatch rumors, which superficially align images and texts, pose particular challenges and threaten online public opinion. Existing multimodal rumor detection methods improve cross modal modeling but suffer from limited feature extraction, noisy alignment, and inflexible fusion strategies, while ignoring external factual evidence necessary for verifying complex rumors. To address these limitations, we propose a multimodal rumor detection model enhanced with external evidence and forgery features. The model uses a ResNet34 visual encoder, a BERT text encoder, and a forgery feature module extracting frequency domain traces and compression artifacts via Fourier transformation. While some existing approaches employ large scale generative vision language models for caption generation, their open ended generation tendency produces verbose and stylistically inconsistent descriptions that introduce semantic noise and risk drifting from actual image content. To overcome this, we adopt BLIP specifically pre trained for vision language alignment which generates concise, image faithful descriptions stylistically closer to news text, serving as a reliable semantic bridge across modalities. A BLIP Driven Semantic Alignment Module jointly optimizes text image and text description contrastive losses, capturing inconsistencies at both visual and semantic levels. A gated adaptive feature scaling fusion mechanism dynamically adjusts multimodal fusion and reduces redundancy. Experiments on Weibo and Twitter datasets demonstrate that our model outperforms mainstream baselines in,recall,and F1 score