# Multimodal Dual Emotion with Fusion of Visual Sentiment for Rumor Detection

arXiv: https://arxiv.org/abs/2204.11515 · HTML: https://mlsanity.com/paper/2204.11515/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 77 references that are in the map.
- First posted on arXiv 25 Apr 2022.

## Builds on (most cited first, 17 of 17)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Semantic Understanding of Scenes through the ADE20K Dataset](https://mlsanity.com/paper/1608.05442/) (2016)
- [Crowdsourcing a Word-Emotion Association Lexicon](https://mlsanity.com/paper/1308.6297/) (2013)
- [Rumor Detection on Social Media with Bi-Directional Graph Convolutional Networks](https://mlsanity.com/paper/2001.06362/) (2020)
- [Detection and Resolution of Rumours in Social Media: A Survey](https://mlsanity.com/paper/1704.00656/) (2017)
- [MDFEND: Multi-domain Fake News Detection](https://mlsanity.com/paper/2201.00987/) (2022)
- [A Survey on Multimodal Disinformation Detection](https://mlsanity.com/paper/2103.12541/) (2021)
- [Robust Image Sentiment Analysis Using Progressively Trained and Domain Transferred Deep Networks](https://mlsanity.com/paper/1509.06041/) (2015)
- [FakeFlow: Fake News Detection by Modeling the Flow of Affective Information](https://mlsanity.com/paper/2101.09810/) (2021)
- [PDANet: Polarity-consistent Deep Attention Network for Fine-grained Visual Emotion Regression](https://mlsanity.com/paper/1909.05693/) (2019)
- [Building Emotional Machines: Recognizing Image Emotions through Deep Neural Networks](https://mlsanity.com/paper/1705.07543/) (2017)
- [Automatic Rumor Detection on Microblogs: A Survey](https://mlsanity.com/paper/1807.03505/) (2018)
- [From Pixels to Sentiment: Fine-tuning CNNs for Visual Sentiment Prediction](https://mlsanity.com/paper/1604.03489/) (2016)
- [VRoC: Variational Autoencoder-aided Multi-task Rumor Classifier Based on Text](https://mlsanity.com/paper/2102.00816/) (2021)
- [Image color transfer to evoke different emotions based on color combinations](https://mlsanity.com/paper/1307.3581/) (2013)
- [On the Role of Images for Analyzing Claims in Social Media](https://mlsanity.com/paper/2103.09602/) (2021)

## Built on by (most cited first, 3 of 3)

- [Mamba-Enhanced Text-Audio-Video Alignment Network for Emotion Recognition in Conversations](https://mlsanity.com/paper/2409.05243/) (2024)
- [Rumor Detection by Multi-task Suffix Learning based on Time-series Dual Sentiments](https://mlsanity.com/paper/2502.14383/) (2025)
- [Region-enhanced Deep Graph Convolutional Networks for Rumor Detection](https://mlsanity.com/paper/2206.07665/) (2022)

## Abstract (arXiv)

In recent years, rumors have had a devastating impact on society, making rumor detection a significant challenge. However, the studies on rumor detection ignore the intense emotions of images in the rumor content. This paper verifies that the image emotion improves the rumor detection efficiency. A Multimodal Dual Emotion feature in rumor detection, which consists of visual and textual emotions, is proposed. To the best of our knowledge, this is the first study which uses visual emotion in rumor detection. The experiments on real datasets verify that the proposed features outperform the state-of-the-art sentiment features, and can be extended in rumor detectors while improving their performance.