# Multimodal Misinformation Detection Using Early Fusion of Linguistic, Visual, and Social Features

arXiv: https://arxiv.org/abs/2507.01984 · HTML: https://mlsanity.com/paper/2507.01984/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 36 references that are in the map.
- First posted on arXiv 26 Jun 2025.

## Builds on (most cited first, 8 of 8)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [Exploiting Multi-domain Visual Information for Fake News Detection](https://mlsanity.com/paper/1908.04472/) (2019)
- [BotOrNot: A System to Evaluate Social Bots](https://mlsanity.com/paper/1602.00975/) (2016)
- [An Exploratory Study of COVID-19 Misinformation on Twitter](https://mlsanity.com/paper/2005.05710/) (2020)
- [In Search of Credible News](https://mlsanity.com/paper/1911.08125/) (2019)
- [Too Little, Too Late: Moderation of Misinformation around the Russo-Ukrainian Conflict](https://mlsanity.com/paper/2502.18500/) (2025)
- [Impact of Online Health Awareness Campaign: Case of National Eating Disorders Association](https://mlsanity.com/paper/2010.05557/) (2020)

## Abstract (arXiv)

Amid a tidal wave of misinformation flooding social media during elections and crises, extensive research has been conducted on misinformation detection, primarily focusing on text-based or image-based approaches. However, only a few studies have explored multimodal feature combinations, such as integrating text and images for building a classification model to detect misinformation. This study investigates the effectiveness of different multimodal feature combinations, incorporating text, images, and social features using an early fusion approach for the classification model. This study analyzed 1,529 tweets containing both text and images during the COVID-19 pandemic and election periods collected from Twitter (now X). A data enrichment process was applied to extract additional social features, as well as visual features, through techniques such as object detection and optical character recognition (OCR). The results show that combining unsupervised and supervised machine learning models improves classification performance by 15% compared to unimodal models and by 5% compared to bimodal models. Additionally, the study analyzes the propagation patterns of misinformation based on the characteristics of misinformation tweets and the users who disseminate them.