mlsanityData to 7 Oct 2026

A Baseline Multimodal Approach to Emotion Recognition in Conversations

arXiv 2602.00914 ↗First posted 31 Jan 2026cs.CL cs.AI cs.CYcited by 0

Not on the map. Why?

Abstract (arXiv)

We present a lightweight multimodal baseline for emotion recognition in conversations using the SemEval-2024 Task 3 dataset built from the sitcom Friends. The goal of this report is not to propose a novel state-of-the-art method, but to document an accessible reference implementation that combines (i) a transformer-based text classifier and (ii) a self-supervised speech representation model, with a simple late-fusion ensemble. We report the baseline setup and empirical results obtained under a limited training protocol, highlighting when multimodal fusion improves over unimodal models. This preprint is provided for transparency and to support future, more rigorous comparisons.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 19 of its 49 references that are in the map. First posted on arXiv 31 Jan 2026.

Builds on

19 of 49 references.

References · 19

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  4. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  5. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  6. 2017Multimodal Machine Learning: A Survey and Taxonomycited by 1,004
  7. 2017Tensor Fusion Network for Multimodal Sentiment Analysiscited by 429
  8. 2019Emotion Recognition in Conversation: Research Challenges, Datasets, and Recent Advancescited by 118
  9. 2024Efficient Multimodal Large Language Models: A Surveycited by 95
  10. 2018Multimodal Sentiment Analysis using Hierarchical Fusion with Context Modelingcited by 39
  11. 2024SemEval-2024 Task 3: Multimodal Emotion Cause Analysis in Conversationscited by 11
  12. 2022Bias and Fairness on Multimodal Emotion Detection Algorithmscited by 10
7 more
  1. 2018Emotion Detection in Text: a Reviewcited by 9
  2. 2023A Comprehensive Survey on Affective Computing; Challenges, Trends, Applications, and Future Directionscited by 7
  3. 2021Emotion Dynamics Modeling via BERTcited by 5
  4. 2020Multi-Modal Emotion Detection with Transfer Learningcited by 4
  5. 2021Classification of Emotions and Evaluation of Customer Satisfaction from Speech in Real World Acoustic Environmentscited by 3
  6. 2021Linguistic and Gender Variation in Speech Emotion Recognition using Spectral Featurescited by 2
  7. 2022Do you know "saudade"? The importance of a cultural and language-based emotion approach for HCIcited by 1

Built on by

Not cited yet.