Emotion Recognition in Conversation: Research Challenges, Datasets, and Recent Advances
Not on the map. Why?
Abstract (arXiv)
Emotion is intrinsic to humans and consequently emotion understanding is a key part of human-like artificial intelligence (AI). Emotion recognition in conversation (ERC) is becoming increasingly popular as a new research frontier in natural language processing (NLP) due to its ability to mine opinions from the plethora of publicly available conversational data in platforms such as Facebook, Youtube, Reddit, Twitter, and others. Moreover, it has potential applications in health-care systems (as a tool for psychological analysis), education (understanding student frustration) and more. Additionally, ERC is also extremely important for generating emotion-aware dialogues that require an understanding of the user's emotions. Catering to these needs calls for effective and scalable conversational emotion-recognition algorithms. However, it is a strenuous problem to solve because of several research challenges. In this paper, we discuss these challenges and shed light on the recent research in this field. We also describe the drawbacks of these approaches and discuss the reasons why they fail to successfully overcome the research challenges in ERC.
Key facts. Cited by 119 papers in the map (data through 14 Oct 2026). First posted on arXiv 8 May 2019.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 15; 2022: 20; 2023: 31; 2024: 15; 2025: 19; 2026: 19.
119 papers.
Sort:
By area:
- 2021Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotationscited by 314
- 2021Directed Acyclic Graph Network for Conversational Emotion Recognitioncited by 97
- 2022GCNet: Graph Completion Network for Incomplete Multimodal Learning in Conversationcited by 87
- 2023Is ChatGPT Equipped with Emotional Dialogue Capabilities?cited by 54
- 2023Towards Interpretable Mental Health Analysis with Large Language Modelscited by 53
- 2022M3ED: Multi-modal Multi-scene Multi-label Emotional Dialogue Databasecited by 50
- 2022M2FNet: Multi-modal Fusion Network for Emotion Recognition in Conversationcited by 47
- 2021CoMPM: Context Modeling with Speaker's Pre-trained Memory Tracking for Emotion Recognition in Conversationcited by 35
- 2025EmoBench-M: Benchmarking Emotional Intelligence for Multimodal Large Language Modelscited by 32
- 2022Supervised Prototypical Contrastive Learning for Emotion Recognition in Conversationcited by 31
- 2022WOODS: Benchmarks for Out-of-Distribution Generalization in Time Seriescited by 30
- 2022COGMEN: COntextualized GNN based Multimodal Emotion recognitioNcited by 29
38 more
- 2021Ruddit: Norms of Offensiveness for English Reddit Commentscited by 27
- 2021Hybrid Curriculum Learning for Emotion Recognition in Conversationcited by 25
- 2024Affective Computing in the Era of Large Language Models: A Survey from the NLP Perspectivecited by 25
- 2023Supervised Adversarial Contrastive Learning for Emotion Recognition in Conversationscited by 19
- 2023Conversation Understanding using Relational Temporal Graph Neural Networks with Auxiliary Cross-Modality Interactioncited by 18
- 2021A Survey on Deep Reinforcement Learning for Audio-Based Applicationscited by 17
- 2021Towards an Online Empathetic Chatbot with Emotion Causescited by 16
- 2021Dyadformer: A Multi-modal Transformer for Long-Range Modeling of Dyadic Interactionscited by 16
- 2023Cluster-Level Contrastive Learning for Emotion Recognition in Conversationscited by 14
- 2023Joyful: Joint Modality Fusion and Graph Contrastive Learning for Multimodal Emotion Recognitioncited by 14
- 2022Deep Emotion Recognition in Textual Conversations: A Surveycited by 13
- 2024Exploring the Efficacy of Large Language Models in Summarizing Mental Health Counseling Sessions: A Benchmark Studycited by 13
- 2024TelME: Teacher-leading Multimodal Fusion Network for Emotion Recognition in Conversationcited by 12
- 2024SemEval-2024 Task 3: Multimodal Emotion Cause Analysis in Conversationscited by 12
- 2023HCAM -- Hierarchical Cross Attention Model for Multi-modal Emotion Recognitioncited by 11
- 2023Bias in Emotion Recognition with ChatGPTcited by 11
- 2025Multimodal Emotion Recognition in Conversations: A Survey of Methods, Trends, Challenges and Prospectscited by 11
- 2021Using Knowledge-Embedded Attention to Augment Pre-trained Language Models for Fine-Grained Emotion Recognitioncited by 10
- 2021Shapes of Emotions: Multimodal Emotion Recognition in Conversations via Emotion Shiftscited by 9
- 2021S+PAGE: A Speaker and Position-Aware Graph Neural Network Model for Emotion Recognition in Conversationcited by 9
- 2022ArmanEmo: A Persian Dataset for Text-based Emotion Detectioncited by 9
- 2023BERT-ERC: Fine-tuning BERT is Enough for Emotion Recognition in Conversationcited by 9
- 2023EmoUS: Simulating User Emotions in Task-Oriented Dialoguescited by 9
- 2025Emergence of Hierarchical Emotion Organization in Large Language Modelscited by 9
- 2021Uncovering the Limits of Text-based Emotion Detectioncited by 8
- 2021KNOT: Knowledge Distillation using Optimal Transport for Solving NLP Taskscited by 8
- 2022M2R2: Missing-Modality Robust emotion Recognition framework with iterative data augmentationcited by 8
- 2022Speaker-Guided Encoder-Decoder Framework for Emotion Recognition in Conversationcited by 8
- 2025DeepDialogue: A Multi-Turn Emotionally-Rich Spoken Dialogue Datasetcited by 8
- 2023EmotionIC: emotional inertia and contagion-driven dependency modeling for emotion recognition in conversationcited by 7
- 2023A Comprehensive Survey on Affective Computing; Challenges, Trends, Applications, and Future Directionscited by 7
- 2022TSAM: A Two-Stream Attention Model for Causal Emotion Entailmentcited by 6
- 2024Empathy Through Multimodality in Conversational Interfacescited by 6
- 2025Akan Cinematic Emotions (ACE): A Multimodal Multi-party Dataset for Emotion Recognition in Movie Dialoguescited by 6
- 2021Emotion Dynamics Modeling via BERTcited by 5
- 2023Disentangled Variational Autoencoder for Emotion Recognition in Conversationscited by 5
- 2023A Survey of Personality, Persona, and Profile in Conversational Agents and Chatbotscited by 5
- 2024Stephanie: Step-by-Step Dialogues for Mimicking Human Interactions in Social Conversationscited by 5
50 most cited of 119 All 119 (JSON)
- 2026Multimodal Emotion Recognition in Conversations via Class-Wise Adaptive Modality Fusion and Affective Geometry9 Sep 2026
- 2026Emo-DVS: A Multimodal Benchmark for Privacy-Aware Emotion Recognition with Event Cameras7 Sep 2026
- 2026SAGE: A Hierarchical Framework for Evaluating Interpretive Literary Quality in Narratives6 Sep 2026
- 2026When Text Misleads: Inconsistent-Aware Reasoning for Audio-Grounded Dialogue27 Aug 2026
- 2026Human-Inspired Social Engagement Analysis via Interpretable Mutual Visual Attention25 Aug 2026
- 2026DiaRelay: Relaying Dialogue Context with a Constant-Size Memory for Emotion Recognition in Conversation24 Aug 2026
- 2026Dual-Scale State-Space Modeling with Speaker-Wise Dynamic CRF for Speech Emotion Recognition in Conversation23 Aug 2026
- 2026Do SpeechLMs Hear Their Own Opinions? Diagnosing and Mitigating Previous-Belief Contamination in Streaming Emotion Understanding21 Aug 2026
- 2026Long-term Measurements: Towards a Longitudinal Understanding of Human-AI Interactions3 Aug 2026
- 2026AtmosERC: Modeling Dialogue-Level Affective Atmosphere for Emotion Recognition in Conversation29 Jul 2026
- 2026EmoEUS: Uncertainty Supervision for Multimodal Emotion Recognition in Conversation19 Jul 2026
- 2026EII-SCL: Harnessing Emotional Inertia for Multimodal Emotion Recognition in Conversation19 Jul 2026
38 more
- 2026SCoPE: Shift-Aware Speaker-Conditioned Priors for Emotion Recognition in Conversations13 May 2026
- 2026Multimodal Hidden Markov Models for Persistent Emotional State Tracking13 May 2026
- 2026Caring Without Feeling: Affective Dynamics as the Control Layer of Human-AI Agent Collaboration8 May 2026
- 2026SAGE: Hierarchical LLM-Based Literary Evaluation through Ontology-Grounded Interpretive Dimensions8 May 2026
- 2026EmoTransCap: Dataset and Pipeline for Emotion Transition-Aware Speech Captioning in Discourses29 Apr 2026
- 2026A Mixture-of-Experts Model for Multimodal Emotion Recognition in Conversations26 Feb 2026
- 2026A Baseline Multimodal Approach to Emotion Recognition in Conversations31 Jan 2026
- 2025Cross-Space Synergy: A Unified Framework for Multimodal Emotion Recognition in Conversation3 Dec 2025
- 2025A Customer Journey in the Land of Oz: Leveraging the Wizard of Oz Technique to Model Emotions in Customer Service Interactions26 Nov 2025
- 2025Emotion Recognition in Multi-Speaker Conversations through Speaker Identification, Knowledge Distillation, and Hierarchical Fusion5 Nov 2025
- 2025ERFC: Happy Customers with Emotion Recognition and Forecasting in Conversation in Call Centers17 Sep 2025
- 2025Emotionally-Aware Agents for Dispute Resolution28 Aug 2025
- 2025Speech Emotion Recognition via Entropy-Aware Score Selection28 Aug 2025
- 2025Can Large Language Models Generate Effective Datasets for Emotion Recognition in Conversations?7 Aug 2025
- 2025Grounding Emotion Recognition with Visual Prototypes: VEGA -- Revisiting CLIP in MERC6 Aug 2025
- 2025Emergence of Hierarchical Emotion Organization in Large Language Models12 Jul 2025
- 2025Multimodal Emotion Recognition in Conversations: A Survey of Methods, Trends, Challenges and Prospects26 May 2025
- 2025DeepDialogue: A Multi-Turn Emotionally-Rich Spoken Dialogue Dataset26 May 2025
- 2025Unimodal-driven Distillation in Multimodal Emotion Recognition with Dynamic Fusion31 Mar 2025
- 2025Hybrid Emotion Recognition: Enhancing Customer Interactions Through Acoustic and Textual Analysis27 Mar 2025
- 2025GatedxLSTM: A Multimodal Affective Computing Approach for Emotion Recognition in Conversations26 Mar 2025
- 2025CALLM: Understanding Cancer Survivors' Emotions and Intervention Opportunities via Mobile Diaries and Context-Aware Language Models12 Mar 2025
- 2025Qieemo: Speech Is All You Need in the Emotion Recognition in Conversations5 Mar 2025
- 2025Akan Cinematic Emotions (ACE): A Multimodal Multi-party Dataset for Emotion Recognition in Movie Dialogues16 Feb 2025
- 2025EmoBench-M: Benchmarking Emotional Intelligence for Multimodal Large Language Models6 Feb 2025
- 2025AIMA at SemEval-2024 Task 10: History-Based Emotion Recognition in Hindi-English Code-Mixed Conversations19 Jan 2025
- 2024Spatio-Temporal Fuzzy-oriented Multi-Modal Meta-Learning for Fine-grained Emotion Recognition18 Dec 2024
- 2024HyperSMOTE: A Hypergraph-based Oversampling Approach for Imbalanced Node Classifications9 Sep 2024
- 2024Tracking Emotional Dynamics in Chat Conversations: A Hybrid Approach using DistilBERT and Emoji Sentiment Analysis3 Aug 2024
- 2024Affective Computing in the Era of Large Language Models: A Survey from the NLP Perspective30 Jul 2024
- 2024Stephanie: Step-by-Step Dialogues for Mimicking Human Interactions in Social Conversations4 Jul 2024
- 2024ConText at WASSA 2024 Empathy and Personality Shared Task: History-Dependent Embedding Utterance Representations for Empathy and Emotion Prediction in Conversations4 Jul 2024
- 2024Emotion Identification for French in Written Texts: Considering their Modes of Expression as a Step Towards Text Complexity Analysis23 May 2024
- 2024SemEval-2024 Task 3: Multimodal Emotion Cause Analysis in Conversations19 May 2024
- 2024Empathy Through Multimodality in Conversational Interfaces8 May 2024
- 2024Personality-affected Emotion Generation in Dialog Systems3 Apr 2024
- 2024Exploring the Efficacy of Large Language Models in Summarizing Mental Health Counseling Sessions: A Benchmark Study29 Feb 2024
- 2024Social Orientation: A New Feature for Dialogue Analysis26 Feb 2024
50 newest of 119 All 119 (JSON)
- 2021Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotationscited by 314
- 2021Directed Acyclic Graph Network for Conversational Emotion Recognitioncited by 97
- 2022GCNet: Graph Completion Network for Incomplete Multimodal Learning in Conversationcited by 87
- 2023Is ChatGPT Equipped with Emotional Dialogue Capabilities?cited by 54
- 2023Towards Interpretable Mental Health Analysis with Large Language Modelscited by 53
- 2022M3ED: Multi-modal Multi-scene Multi-label Emotional Dialogue Databasecited by 50
- 2022M2FNet: Multi-modal Fusion Network for Emotion Recognition in Conversationcited by 47
- 2021CoMPM: Context Modeling with Speaker's Pre-trained Memory Tracking for Emotion Recognition in Conversationcited by 35
- 2022Supervised Prototypical Contrastive Learning for Emotion Recognition in Conversationcited by 31
- 2022WOODS: Benchmarks for Out-of-Distribution Generalization in Time Seriescited by 30
- 2022COGMEN: COntextualized GNN based Multimodal Emotion recognitioNcited by 29
- 2021Ruddit: Norms of Offensiveness for English Reddit Commentscited by 27
12 most cited of 110 All 110 (JSON)
- 2026Multimodal Emotion Recognition in Conversations via Class-Wise Adaptive Modality Fusion and Affective Geometry9 Sep 2026
- 2026Emo-DVS: A Multimodal Benchmark for Privacy-Aware Emotion Recognition with Event Cameras7 Sep 2026
- 2026SAGE: A Hierarchical Framework for Evaluating Interpretive Literary Quality in Narratives6 Sep 2026
- 2026Human-Inspired Social Engagement Analysis via Interpretable Mutual Visual Attention25 Aug 2026
- 2026DiaRelay: Relaying Dialogue Context with a Constant-Size Memory for Emotion Recognition in Conversation24 Aug 2026
- 2026Dual-Scale State-Space Modeling with Speaker-Wise Dynamic CRF for Speech Emotion Recognition in Conversation23 Aug 2026
- 2026Do SpeechLMs Hear Their Own Opinions? Diagnosing and Mitigating Previous-Belief Contamination in Streaming Emotion Understanding21 Aug 2026
- 2026Long-term Measurements: Towards a Longitudinal Understanding of Human-AI Interactions3 Aug 2026
- 2026AtmosERC: Modeling Dialogue-Level Affective Atmosphere for Emotion Recognition in Conversation29 Jul 2026
- 2026EmoEUS: Uncertainty Supervision for Multimodal Emotion Recognition in Conversation19 Jul 2026
- 2026EII-SCL: Harnessing Emotional Inertia for Multimodal Emotion Recognition in Conversation19 Jul 2026
- 2026SCoPE: Shift-Aware Speaker-Conditioned Priors for Emotion Recognition in Conversations13 May 2026
12 newest of 110 All 110 (JSON)