# Exploring the Role of Audio in Multimodal Misinformation Detection

arXiv: https://arxiv.org/abs/2408.12558 · HTML: https://mlsanity.com/paper/2408.12558/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Aug 2024.

## Built on by (most cited first, 5 of 5)

- [Multimodal Fact-Checking with Vision Language Models: A Probing Classifier based Solution with Embedding Strategies](https://mlsanity.com/paper/2412.05155/) (2024)
- [Deconfounded Reasoning for Multimodal Fake News Detection via Causal Intervention](https://mlsanity.com/paper/2504.09163/) (2025)
- [Enhancing Fake News Video Detection via LLM-Driven Creative Process Simulation](https://mlsanity.com/paper/2510.04024/) (2025)
- [Exploring Modality Disruption in Multimodal Fake News Detection](https://mlsanity.com/paper/2504.09154/) (2025)
- [When Misinformation Speaks and Converses: Rethinking Fact-Checking in Audio Platforms](https://mlsanity.com/paper/2604.16767/) (2026)

## Abstract (arXiv)

With the rapid development of deepfake technology, especially the deep audio fake technology, misinformation detection on the social media scene meets a great challenge. Social media data often contains multimodal information which includes audio, video, text, and images. However, existing multimodal misinformation detection methods tend to focus only on some of these modalities, failing to comprehensively address information from all modalities. To comprehensively address the various modal information that may appear on social media, this paper constructs a comprehensive multimodal misinformation detection framework. By employing corresponding neural network encoders for each modality, the framework can fuse different modality information and support the multimodal misinformation detection task. Based on the constructed framework, this paper explores the importance of the audio modality in multimodal misinformation detection tasks on social media. By adjusting the architecture of the acoustic encoder, the effectiveness of different acoustic feature encoders in the multimodal misinformation detection tasks is investigated. Furthermore, this paper discovers that audio and video information must be carefully aligned, otherwise the misalignment across different audio and video modalities can severely impair the model performance.