# Multimodal Learning for Fake News Detection in Short Videos Using Linguistically Verified Data and Heterogeneous Modality Fusion

arXiv: https://arxiv.org/abs/2509.15578 · HTML: https://mlsanity.com/paper/2509.15578/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 50 references that are in the map.
- First posted on arXiv 19 Sep 2025.

## Builds on (most cited first, 20 of 20)

- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [Video Swin Transformer](https://mlsanity.com/paper/2106.13230/) (2021)
- [Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation](https://mlsanity.com/paper/2211.06687/) (2022)
- [Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey](https://mlsanity.com/paper/1902.06162/) (2019)
- [DistilHuBERT: Speech Representation Learning by Layer-wise Distillation of Hidden-unit BERT](https://mlsanity.com/paper/2110.01900/) (2021)
- [Multimodal Intelligence: Representation Learning, Information Fusion, and Applications](https://mlsanity.com/paper/1911.03977/) (2019)
- [Self-supervised Video Transformer](https://mlsanity.com/paper/2112.01514/) (2021)
- [Embracing Domain Differences in Fake News: Cross-domain Fake News Detection using Multi-modal Data](https://mlsanity.com/paper/2102.06314/) (2021)
- [Improving Fake News Detection by Using an Entity-enhanced Framework to Fuse Diverse Multimodal Clues](https://mlsanity.com/paper/2108.10509/) (2021)
- [FakeSV: A Multimodal Benchmark with Rich Social Context for Fake News Detection on Short Video Platforms](https://mlsanity.com/paper/2211.10973/) (2022)
- [FakingRecipe: Detecting Fake News on Short Video Platforms from the Perspective of Creative Process](https://mlsanity.com/paper/2407.16670/) (2024)
- [Towards Automatic Detection of Misinformation in Online Medical Videos](https://mlsanity.com/paper/1909.01543/) (2019)
- [COVID-VTS: Fact Extraction and Verification on Short Video Platforms](https://mlsanity.com/paper/2302.07919/) (2023)
- [Combating Online Misinformation Videos: Characterization, Detection, and Future Directions](https://mlsanity.com/paper/2302.03242/) (2023)
- [Misleading Metadata Detection on YouTube](https://mlsanity.com/paper/1901.08759/) (2019)
- [Misinformation Detection on YouTube Using Video Captions](https://mlsanity.com/paper/2107.00941/) (2021)

## Abstract (arXiv)

The rapid proliferation of short video platforms has necessitated advanced methods for detecting fake news. This need arises from the widespread influence and ease of sharing misinformation, which can lead to significant societal harm. Current methods often struggle with the dynamic and multimodal nature of short video content. This paper presents HFN, Heterogeneous Fusion Net, a novel multimodal framework that integrates video, audio, and text data to evaluate the authenticity of short video content. HFN introduces a Decision Network that dynamically adjusts modality weights during inference and a Weighted Multi-Modal Feature Fusion module to ensure robust performance even with incomplete data. Additionally, we contribute a comprehensive dataset VESV (VEracity on Short Videos) specifically designed for short video fake news detection. Experiments conducted on the FakeTT and newly collected VESV datasets demonstrate improvements of 2.71% and 4.14% in Marco F1 over state-of-the-art methods. This work establishes a robust solution capable of effectively identifying fake news in the complex landscape of short video platforms, paving the way for more reliable and comprehensive approaches in combating misinformation.