# A Baseline Multimodal Approach to Emotion Recognition in Conversations

arXiv: https://arxiv.org/abs/2602.00914 · HTML: https://mlsanity.com/paper/2602.00914/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 49 references that are in the map.
- First posted on arXiv 31 Jan 2026.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [Multimodal Machine Learning: A Survey and Taxonomy](https://mlsanity.com/paper/1705.09406/) (2017)
- [Tensor Fusion Network for Multimodal Sentiment Analysis](https://mlsanity.com/paper/1707.07250/) (2017)
- [Emotion Recognition in Conversation: Research Challenges, Datasets, and Recent Advances](https://mlsanity.com/paper/1905.02947/) (2019)
- [Efficient Multimodal Large Language Models: A Survey](https://mlsanity.com/paper/2405.10739/) (2024)
- [Multimodal Sentiment Analysis using Hierarchical Fusion with Context Modeling](https://mlsanity.com/paper/1806.06228/) (2018)
- [SemEval-2024 Task 3: Multimodal Emotion Cause Analysis in Conversations](https://mlsanity.com/paper/2405.13049/) (2024)
- [Emotion Detection in Text: a Review](https://mlsanity.com/paper/1806.00674/) (2018)
- [Bias and Fairness on Multimodal Emotion Detection Algorithms](https://mlsanity.com/paper/2205.08383/) (2022)
- [A Comprehensive Survey on Affective Computing; Challenges, Trends, Applications, and Future Directions](https://mlsanity.com/paper/2305.07665/) (2023)
- [Emotion Dynamics Modeling via BERT](https://mlsanity.com/paper/2104.07252/) (2021)
- [Multi-Modal Emotion Detection with Transfer Learning](https://mlsanity.com/paper/2011.07065/) (2020)
- [Classification of Emotions and Evaluation of Customer Satisfaction from Speech in Real World Acoustic Environments](https://mlsanity.com/paper/2108.11981/) (2021)
- [Linguistic and Gender Variation in Speech Emotion Recognition using Spectral Features](https://mlsanity.com/paper/2112.09596/) (2021)
- [Do you know "saudade"? The importance of a cultural and language-based emotion approach for HCI](https://mlsanity.com/paper/2204.06119/) (2022)

## Abstract (arXiv)

We present a lightweight multimodal baseline for emotion recognition in conversations using the SemEval-2024 Task 3 dataset built from the sitcom Friends. The goal of this report is not to propose a novel state-of-the-art method, but to document an accessible reference implementation that combines (i) a transformer-based text classifier and (ii) a self-supervised speech representation model, with a simple late-fusion ensemble. We report the baseline setup and empirical results obtained under a limited training protocol, highlighting when multimodal fusion improves over unimodal models. This preprint is provided for transparency and to support future, more rigorous comparisons.