mlsanityData to 7 Oct 2026

Shapes of Emotions: Multimodal Emotion Recognition in Conversations via Emotion Shifts

arXiv 2112.01938 ↗First posted 3 Dec 2021cs.CL cs.AI cs.LGcited by 9

Not on the map. Why?

Abstract (arXiv)

Emotion Recognition in Conversations (ERC) is an important and active research area. Recent work has shown the benefits of using multiple modalities (e.g., text, audio, and video) for the ERC task. In a conversation, participants tend to maintain a particular emotional state unless some stimuli evokes a change. There is a continuous ebb and flow of emotions in a conversation. Inspired by this observation, we propose a multimodal ERC model and augment it with an emotion-shift component that improves performance. The proposed emotion-shift component is modular and can be added to any existing multimodal ERC model (with a few modifications). We experiment with different variants of the model, and results show that the inclusion of emotion shift signal helps the model to outperform existing models for ERC on MOSEI and IEMOCAP datasets.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 24 of its 50 references that are in the map. First posted on arXiv 3 Dec 2021.

Builds on

24 of 50 references.

References · 24

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  5. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  6. 2017Multimodal Machine Learning: A Survey and Taxonomycited by 1,004
  7. 2018DialogueRNN: An Attentive RNN for Emotion Detection in Conversationscited by 216
  8. 2019DialogueGCN: A Graph Convolutional Neural Network for Emotion Recognition in Conversationcited by 184
  9. 2018Memory Fusion Network for Multi-view Sequential Learningcited by 167
  10. 2020COSMIC: COmmonSense knowledge for eMotion Identification in Conversationscited by 135
  11. 2019Emotion Recognition in Conversation: Research Challenges, Datasets, and Recent Advancescited by 118
  12. 2016The emotional arcs of stories are dominated by six basic shapescited by 69
12 more
  1. 2017End-to-End Multimodal Emotion Recognition using Deep Neural Networkscited by 69
  2. 2019Affect-Driven Dialog Generationcited by 50
  3. 2020A Transformer-based joint-encoding for Emotion Recognition and Sentiment Analysiscited by 46
  4. 2018Multimodal Sentiment Analysis using Hierarchical Fusion with Context Modelingcited by 39
  5. 2020Multilogue-Net: A Context Aware RNN for Multi-modal Emotion Detection and Sentiment Analysis in Conversationcited by 27
  6. 2018Multimodal Sentiment Analysis: Addressing Key Issues and Setting up the Baselinescited by 23
  7. 2020Adapting a Language Model for Controlled Affective Text Generationcited by 22
  8. 2021An End-to-End Network for Emotion-Cause Pair Extractioncited by 15
  9. 2018Deep learning for affective computing: text-based emotion recognition in decision supportcited by 14
  10. 2021Fine-Grained Emotion Prediction by Modeling Emotion Definitionscited by 9
  11. 2019Domain Authoring Assistant for Intelligent Virtual Agentscited by 5
  12. 2017Improving speech recognition by revising gated recurrent unitscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 3; 2024: 3; 2025: 0; 2026: 1.

9 papers.

  1. 2022COGMEN: COntextualized GNN based Multimodal Emotion recognitioNcited by 29
  2. 2023CFN-ESA: A Cross-Modal Fusion Network with Emotion-Shift Awareness for Dialogue Emotion Recognitioncited by 14
  3. 2023Revisiting Disentanglement and Fusion on Modality and Context in Conversational Multimodal Emotion Recognitioncited by 13
  4. 2022Multi-Task Learning Framework for Extracting Emotion Cause Span and Entailment in Conversationscited by 3
  5. 2023Dynamic Causal Disentanglement Model for Dialogue Emotion Detectioncited by 1
  6. 2024AMuSE: Adaptive Multimodal Analysis for Speaker Emotion Recognition in Group Conversationscited by 1
  7. 2024AM^2-EmoJE: Adaptive Missing-Modality Emotion Recognition in Conversation via Joint Embedding Learningcited by 0
  8. 2024IITK at SemEval-2024 Task 10: Who is the speaker? Improving Emotion Recognition and Flip Reasoning in Conversations via Speaker Embeddingscited by 0
  9. 2026Multimodal Emotion Recognition in Conversations via Class-Wise Adaptive Modality Fusion and Affective Geometrycited by 0