# CroMe: Multimodal Fake News Detection using Cross-Modal Tri-Transformer and Metric Learning

arXiv: https://arxiv.org/abs/2501.12422 · HTML: https://mlsanity.com/paper/2501.12422/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 45 references that are in the map.
- First posted on arXiv 21 Jan 2025.

## Builds on (most cited first, 17 of 17)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Fake News Detection on Social Media: A Data Mining Perspective](https://mlsanity.com/paper/1708.01967/) (2017)
- [FakeNewsNet: A Data Repository with News Content, Social Context and Spatialtemporal Information for Studying Fake News on Social Media](https://mlsanity.com/paper/1809.01286/) (2018)
- [No Fuss Distance Metric Learning using Proxies](https://mlsanity.com/paper/1703.07464/) (2017)
- [Proxy Anchor Loss for Deep Metric Learning](https://mlsanity.com/paper/2003.13911/) (2020)
- [MDFEND: Multi-domain Fake News Detection](https://mlsanity.com/paper/2201.00987/) (2022)
- [Cosine Normalization: Using Cosine Similarity Instead of Dot Product in Neural Networks](https://mlsanity.com/paper/1702.05870/) (2017)
- [Multimodal Fake News Detection via CLIP-Guided Learning](https://mlsanity.com/paper/2205.14304/) (2022)
- [Call Attention to Rumors: Deep Attention Based Recurrent Neural Networks for Early Rumor Detection](https://mlsanity.com/paper/1704.05973/) (2017)
- [Leveraging Multi-Source Weak Social Supervision for Early Detection of Fake News](https://mlsanity.com/paper/2004.01732/) (2020)
- [Explainable Tsetlin Machine framework for fake news detection with credibility score assessment](https://mlsanity.com/paper/2105.09114/) (2021)
- [TT-BLIP: Enhancing Fake News Detection Using BLIP and Tri-Transformer](https://mlsanity.com/paper/2403.12481/) (2024)

## Built on by (most cited first, 2 of 2)

- [UNITE-FND: Reframing Multimodal Fake News Detection through Unimodal Scene Translation](https://mlsanity.com/paper/2502.11132/) (2025)
- [Towards Explainable Bilingual Multimodal Misinformation Detection and Localization](https://mlsanity.com/paper/2506.22930/) (2025)

## Abstract (arXiv)

Multimodal Fake News Detection has received increasing attention recently. Existing methods rely on independently encoded unimodal data and overlook the advantages of capturing intra-modality relationships and integrating inter-modal similarities using advanced techniques. To address these issues, Cross-Modal Tri-Transformer and Metric Learning for Multimodal Fake News Detection (CroMe) is proposed. CroMe utilizes Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models (BLIP2) as encoders to capture detailed text, image and combined image-text representations. The metric learning module employs a proxy anchor method to capture intra-modality relationships while the feature fusion module uses a Cross-Modal and Tri-Transformer for effective integration. The final fake news detector processes the fused features through a classifier to predict the authenticity of the content. Experiments on datasets show that CroMe excels in multimodal fake news detection.