# Multimodal Sentiment Analysis: Addressing Key Issues and Setting up the Baselines

arXiv: https://arxiv.org/abs/1803.07427 · HTML: https://mlsanity.com/paper/1803.07427/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Mar 2018.

## Built on by (most cited first, 23 of 23)

- [Speech Emotion Recognition using Self-Supervised Features](https://mlsanity.com/paper/2202.03896/) (2022)
- [Make Acoustic and Visual Cues Matter: CH-SIMS v2.0 Dataset and AV-Mixup Consistent Module](https://mlsanity.com/paper/2209.02604/) (2022)
- [Tag-assisted Multimodal Sentiment Analysis under Uncertain Missing Modalities](https://mlsanity.com/paper/2204.13707/) (2022)
- [Group Gated Fusion on Attention-based Bidirectional Alignment for Multimodal Emotion Recognition](https://mlsanity.com/paper/2201.06309/) (2022)
- [MultiModN- Multimodal, Multi-Task, Interpretable Modular Networks](https://mlsanity.com/paper/2309.14118/) (2023)
- [Emotion fusion for mental illness detection from social media: A survey](https://mlsanity.com/paper/2304.09493/) (2023)
- [Estimating the Uncertainty in Emotion Attributes using Deep Evidential Regression](https://mlsanity.com/paper/2306.06760/) (2023)
- [Shapes of Emotions: Multimodal Emotion Recognition in Conversations via Emotion Shifts](https://mlsanity.com/paper/2112.01938/) (2021)
- [Multistage linguistic conditioning of convolutional layers for speech emotion recognition](https://mlsanity.com/paper/2110.06650/) (2021)
- [Estimating the Uncertainty in Emotion Class Labels with Utterance-Specific Dirichlet Priors](https://mlsanity.com/paper/2203.04443/) (2022)
- [Climate and Weather: Inspecting Depression Detection via Emotion Recognition](https://mlsanity.com/paper/2204.14099/) (2022)
- [MMER: Multimodal Multi-task Learning for Speech Emotion Recognition](https://mlsanity.com/paper/2203.16794/) (2022)
- [One-Versus-Others Attention: Scalable Multimodal Integration for Biomedical Data](https://mlsanity.com/paper/2307.05435/) (2023)
- [DravidianMultiModality: A Dataset for Multi-modal Sentiment Analysis in Tamil and Malayalam](https://mlsanity.com/paper/2106.04853/) (2021)
- [CTA-RNN: Channel and Temporal-wise Attention RNN Leveraging Pre-trained ASR Embeddings for Speech Emotion Recognition](https://mlsanity.com/paper/2203.17023/) (2022)
- [Distribution-based Emotion Recognition in Conversation](https://mlsanity.com/paper/2211.04834/) (2022)
- [Sequential Late Fusion Technique for Multi-modal Sentiment Analysis](https://mlsanity.com/paper/2106.11473/) (2021)
- [LETS-C: Leveraging Text Embedding for Time Series Classification](https://mlsanity.com/paper/2407.06533/) (2024)
- [Dialog speech sentiment classification for imbalanced datasets](https://mlsanity.com/paper/2109.07228/) (2021)
- [Exploring a Hybrid Deep Learning Framework to Automatically Discover Topic and Sentiment in COVID-19 Tweets](https://mlsanity.com/paper/2312.01178/) (2023)
- [Incomplete Multi-view Multi-label Classification via a Dual-level Contrastive Learning Framework](https://mlsanity.com/paper/2411.18267/) (2024)
- [Speech Emotion Recognition via Entropy-Aware Score Selection](https://mlsanity.com/paper/2508.20796/) (2025)
- [Teaching AI to Feel: A Collaborative, Full-Body Exploration of Emotive Communication](https://mlsanity.com/paper/2509.22168/) (2025)

## Abstract (arXiv)

We compile baselines, along with dataset split, for multimodal sentiment analysis. In this paper, we explore three different deep-learning based architectures for multimodal sentiment classification, each improving upon the previous. Further, we evaluate these architectures with multiple datasets with fixed train/test partition. We also discuss some major issues, frequently ignored in multimodal sentiment analysis research, e.g., role of speaker-exclusive models, importance of different modalities, and generalizability. This framework illustrates the different facets of analysis to be considered while performing multimodal sentiment analysis and, hence, serves as a new benchmark for future research in this emerging field.