mlsanityData to 14 Oct 2026

Multimodal Emotion Recognition in Conversations: A Survey of Methods, Trends, Challenges and Prospects

arXiv 2505.20511 ↗First posted 26 May 2025cs.CLcited by 11

Not on the map. Why?

Abstract (arXiv)

While text-based emotion recognition methods have achieved notable success, real-world dialogue systems often demand a more nuanced emotional understanding than any single modality can offer. Multimodal Emotion Recognition in Conversations (MERC) has thus emerged as a crucial direction for enhancing the naturalness and emotional understanding of human-computer interaction. Its goal is to accurately recognize emotions by integrating information from various modalities such as text, speech, and visual signals. This survey offers a systematic overview of MERC, including its motivations, core tasks, representative methods, and evaluation strategies. We further examine recent trends, highlight key challenges, and outline future directions. As interest in emotionally intelligent systems grows, this survey provides timely guidance for advancing MERC research.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 47 of its 123 references that are in the map. First posted on arXiv 26 May 2025.

Builds on

47 of 123 references.

References · 47

  1. 2017Attention Is All You Needcited by 60,361
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  5. 2016Densely Connected Convolutional Networkscited by 6,720
  6. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  7. 2014Convolutional Neural Networks for Sentence Classificationcited by 1,127
  8. 2016Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networkscited by 903
  9. 2018MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversationscited by 643
  10. 2019Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillationcited by 497
  11. 2018Hypergraph Neural Networkscited by 488
  12. 2018DialogueRNN: An Attentive RNN for Emotion Detection in Conversationscited by 218
35 more
  1. 2020COSMIC: COmmonSense knowledge for eMotion Identification in Conversationscited by 136
  2. 2023FourierGNN: Rethinking Multivariate Time Series Forecasting from a Pure Graph Perspectivecited by 127
  3. 2019Emotion Recognition in Conversation: Research Challenges, Datasets, and Recent Advancescited by 119
  4. 2017Emotion Detection on TV Show Transcripts with Sequence-based Convolutional Neural Networkscited by 102
  5. 2019Hierarchical Gating Networks for Sequential Recommendationcited by 101
  6. 2021MMGCN: Multimodal Fusion via Deep Graph Convolution Network for Emotion Recognition in Conversationcited by 99
  7. 2022GCNet: Graph Completion Network for Incomplete Multimodal Learning in Conversationcited by 87
  8. 2018EmotionLines: An Emotion Corpus of Multi-Party Conversationscited by 80
  9. 2024Multimodal Alignment and Fusion: A Surveycited by 74
  10. 2022M3ED: Multi-modal Multi-scene Multi-label Emotional Dialogue Databasecited by 50
  11. 2024Multimodal Fusion on Low-quality Data: A Comprehensive Surveycited by 49
  12. 2022EmoCaps: Emotion Capsule based Model for Conversational Emotion Recognitioncited by 48
  13. 2018Survey on Emotional Body Gesture Recognitioncited by 47
  14. 2022M2FNet: Multi-modal Fusion Network for Emotion Recognition in Conversationcited by 47
  15. 2021Hierarchical Self-supervised Augmented Knowledge Distillationcited by 44
  16. 2022Revisiting Graph Contrastive Learning from the Perspective of Graph Spectrumcited by 43
  17. 2019Real-Time Emotion Recognition via Attention Gated Hierarchical Memory Networkcited by 38
  18. 2024Revisiting Multimodal Emotion Recognition in Conversation from the Perspective of Graph Spectrumcited by 33
  19. 2023DialogueLLM: Context and Emotion Knowledge-Tuned Large Language Models for Emotion Recognition in Conversationscited by 30
  20. 2024Beyond Silent Letters: Amplifying LLMs in Emotion Recognition with Vocal Nuancescited by 27
  21. 2023Conversation Understanding using Relational Temporal Graph Neural Networks with Auxiliary Cross-Modality Interactioncited by 18
  22. 2023UniSA: Unified Generative Framework for Sentiment Analysiscited by 17
  23. 2023CFN-ESA: A Cross-Modal Fusion Network with Emotion-Shift Awareness for Dialogue Emotion Recognitioncited by 14
  24. 2024LaERC-S: Improving LLM-based Emotion Recognition in Conversation with Speaker Characteristicscited by 13
  25. 2024Text-centric Alignment for Multi-Modality Learningcited by 12
  26. 2022Few-shot Multimodal Sentiment Analysis based on Multimodal Probabilistic Fusion Promptscited by 9
  27. 2024Learning Modality Knowledge Alignment for Cross-Modality Transfercited by 8
  28. 2024Graph Neural Networks in EEG-based Emotion Recognition: A Surveycited by 6
  29. 2025Akan Cinematic Emotions (ACE): A Multimodal Multi-party Dataset for Emotion Recognition in Movie Dialoguescited by 6
  30. 2020Multi-Modal Emotion Detection with Transfer Learningcited by 4
  31. 2022ModSelect: Automatic Modality Selection for Synthetic-to-Real Domain Generalizationcited by 4
  32. 2024Toward a Dialogue System Using a Large Language Model to Recognize User Emotions with a Cameracited by 2
  33. 2024Effective Context Modeling Framework for Emotion Recognition in Conversationscited by 2
  34. 2024Smile upon the Face but Sadness in the Eyes: Emotion Recognition based on Facial Expressions and Eye Behaviorscited by 1
  35. 2025Cross-modal Context Fusion and Adaptive Graph Convolutional Network for Multimodal Conversational Emotion Recognitioncited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 9.

11 papers.

  1. 2025Learning More with Less: Self-Supervised Approaches for Low-Resource Speech Emotion Recognitioncited by 2
  2. 2026SCoPE: Shift-Aware Speaker-Conditioned Priors for Emotion Recognition in Conversationscited by 2
  3. 2025Decoding Student Minds: Leveraging Conversational Agents for Psychological and Learning Analysiscited by 0
  4. 2026AMB-DSGDN: Adaptive Modality-Balanced Dynamic Semantic Graph Differential Network for Multimodal Emotion Recognitioncited by 0
  5. 2026CMHL: Contrastive Multi-Head Learning for Emotionally Consistent Text Classificationcited by 0
  6. 2026SURE: Synergistic Uncertainty-aware Reasoning for Multimodal Emotion Recognition in Conversationscited by 0
  7. 2026MISID: A Multimodal Multi-turn Dataset for Complex Intent Recognition in Strategic Deception Gamescited by 0
  8. 2026AffectSeek: Agentic Affective Understanding in Long Videos under Vague User Queriescited by 0
  9. 2026MA-DLE: Speech-based Automatic Depression Level Estimation via Memory Augmentationcited by 0
  10. 2026Empath: Tracing Multi-Level Emotion Dynamics in Crisis Counseling Dialoguescited by 0
  11. 2026GeoMER: A Joint Euclidean-Hyperbolic Geometry Framework for Multimodal Emotion Recognitioncited by 0