# A Self-Learning Multimodal Approach for Fake News Detection

arXiv: https://arxiv.org/abs/2412.05843 · HTML: https://mlsanity.com/paper/2412.05843/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 40 references that are in the map.
- First posted on arXiv 8 Dec 2024.

## Builds on (most cited first, 19 of 19)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models](https://mlsanity.com/paper/2304.10592/) (2023)
- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [CAFE: Catastrophic Data Leakage in Vertical Federated Learning](https://mlsanity.com/paper/2110.15122/) (2021)
- [Exploiting Multi-domain Visual Information for Fake News Detection](https://mlsanity.com/paper/1908.04472/) (2019)
- [Eyes Tell All: Irregular Pupil Shapes Reveal GAN-generated Faces](https://mlsanity.com/paper/2109.00162/) (2021)
- [VERITE: A Robust Benchmark for Multimodal Misinformation Detection Accounting for Unimodal Bias](https://mlsanity.com/paper/2304.14133/) (2023)
- [FKA-Owl: Advancing Multimodal Fake News Detection through Knowledge-Augmented LVLMs](https://mlsanity.com/paper/2403.01988/) (2024)
- [GAN-generated Faces Detection: A Survey and New Perspectives](https://mlsanity.com/paper/2202.07145/) (2022)
- [Robust Attentive Deep Neural Network for Exposing GAN-generated Faces](https://mlsanity.com/paper/2109.02167/) (2021)
- [Adapting Fake News Detection to the Era of Large Language Models](https://mlsanity.com/paper/2311.04917/) (2023)
- [Large Language Model Agent for Fake News Detection](https://mlsanity.com/paper/2405.01593/) (2024)
- [Open-Eye: An Open Platform to Study Human Performance on Identifying AI-Synthesized Faces](https://mlsanity.com/paper/2205.06680/) (2022)
- [The Future of Misinformation Detection: New Perspectives and Trends](https://mlsanity.com/paper/1909.03654/) (2019)
- [Harnessing the Power of Text-image Contrastive Models for Automatic Detection of Online Misinformation](https://mlsanity.com/paper/2304.10249/) (2023)
- [ACM Multimedia Grand Challenge on Detecting Cheapfakes](https://mlsanity.com/paper/2207.14534/) (2022)

## Built on by (most cited first, 3 of 3)

- [Improving Generalization of Medical Image Registration Foundation Model](https://mlsanity.com/paper/2505.06527/) (2025)
- [Robust AI-Generated Face Detection with Imbalanced Data](https://mlsanity.com/paper/2505.02182/) (2025)
- [SynthGuard: An Open Platform for Detecting AI-Generated Multimedia with Multimodal LLMs](https://mlsanity.com/paper/2511.12404/) (2025)

## Abstract (arXiv)

The rapid growth of social media has resulted in an explosion of online news content, leading to a significant increase in the spread of misleading or false information. While machine learning techniques have been widely applied to detect fake news, the scarcity of labeled datasets remains a critical challenge. Misinformation frequently appears as paired text and images, where a news article or headline is accompanied by a related visuals. In this paper, we introduce a self-learning multimodal model for fake news classification. The model leverages contrastive learning, a robust method for feature extraction that operates without requiring labeled data, and integrates the strengths of Large Language Models (LLMs) to jointly analyze both text and image features. LLMs are excel at this task due to their ability to process diverse linguistic data drawn from extensive training corpora. Our experimental results on a public dataset demonstrate that the proposed model outperforms several state-of-the-art classification approaches, achieving over 85% accuracy, precision, recall, and F1-score. These findings highlight the model's effectiveness in tackling the challenges of multimodal fake news detection.