# A Cross-Modal Rumor Detection Scheme via Contrastive Learning by Exploring Text and Image internal Correlations

arXiv: https://arxiv.org/abs/2508.11141 · HTML: https://mlsanity.com/paper/2508.11141/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 49 references that are in the map.
- First posted on arXiv 15 Aug 2025.

## Builds on (most cited first, 13 of 13)

- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Graph Convolutional Networks for Text Classification](https://mlsanity.com/paper/1809.05679/) (2018)
- [How Much Can CLIP Benefit Vision-and-Language Tasks?](https://mlsanity.com/paper/2107.06383/) (2021)
- [Chinese CLIP: Contrastive Vision-Language Pretraining in Chinese](https://mlsanity.com/paper/2211.01335/) (2022)
- [The Fact Extraction and VERification (FEVER) Shared Task](https://mlsanity.com/paper/1811.10971/) (2018)
- [COCO-CN for Cross-Lingual Image Tagging, Captioning and Retrieval](https://mlsanity.com/paper/1805.08661/) (2018)
- [Multimodal Fake News Detection via CLIP-Guided Learning](https://mlsanity.com/paper/2205.14304/) (2022)
- [Towards Propagation Uncertainty: Edge-enhanced Bayesian Graph Convolutional Networks for Rumor Detection](https://mlsanity.com/paper/2107.11934/) (2021)
- [CED: Credible Early Detection of Social Media Rumors](https://mlsanity.com/paper/1811.04175/) (2018)
- [Modeling Conversation Structure and Temporal Dynamics for Jointly Predicting Rumor Stance and Veracity](https://mlsanity.com/paper/1909.08211/) (2019)
- [Inconsistent Matters: A Knowledge-guided Dual-consistency Network for Multi-modal Rumor Detection](https://mlsanity.com/paper/2306.02137/) (2023)
- [ETMA: Efficient Transformer Based Multilevel Attention framework for Multimodal Fake News Detection](https://mlsanity.com/paper/2206.07331/) (2022)
- [Focusing on Relevant Responses for Multi-modal Rumor Detection](https://mlsanity.com/paper/2306.11746/) (2023)

## Abstract (arXiv)

Existing rumor detection methods often neglect the content within images as well as the inherent relationships between contexts and images across different visual scales, thereby resulting in the loss of critical information pertinent to rumor identification. To address these issues, this paper presents a novel cross-modal rumor detection scheme based on contrastive learning, namely the Multi-scale Image and Context Correlation exploration algorithm (MICC). Specifically, we design an SCLIP encoder to generate unified semantic embeddings for text and multi-scale image patches through contrastive pretraining, enabling their relevance to be measured via dot-product similarity. Building upon this, a Cross-Modal Multi-Scale Alignment module is introduced to identify image regions most relevant to the textual semantics, guided by mutual information maximization and the information bottleneck principle, through a Top-K selection strategy based on a cross-modal relevance matrix constructed between the text and multi-scale image patches. Moreover, a scale-aware fusion network is designed to integrate the highly correlated multi-scale image features with global text features by assigning adaptive weights to image regions based on their semantic importance and cross-modal relevance. The proposed methodology has been extensively evaluated on two real-world datasets. The experimental results demonstrate that it achieves a substantial performance improvement over existing state-of-the-art approaches in rumor detection, highlighting its effectiveness and potential for practical applications.