# EmoCaps: Emotion Capsule based Model for Conversational Emotion Recognition

arXiv: https://arxiv.org/abs/2203.13504 · HTML: https://mlsanity.com/paper/2203.13504/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 28 references that are in the map.
- First posted on arXiv 25 Mar 2022.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations](https://mlsanity.com/paper/1810.02508/) (2018)
- [Tensor Fusion Network for Multimodal Sentiment Analysis](https://mlsanity.com/paper/1707.07250/) (2017)
- [A Survey of Transformers](https://mlsanity.com/paper/2106.04554/) (2021)
- [DialogueRNN: An Attentive RNN for Emotion Detection in Conversations](https://mlsanity.com/paper/1811.00405/) (2018)
- [DialogueGCN: A Graph Convolutional Neural Network for Emotion Recognition in Conversation](https://mlsanity.com/paper/1908.11540/) (2019)
- [COSMIC: COmmonSense knowledge for eMotion Identification in Conversations](https://mlsanity.com/paper/2010.02795/) (2020)
- [MMGCN: Multimodal Fusion via Deep Graph Convolution Network for Emotion Recognition in Conversation](https://mlsanity.com/paper/2107.06779/) (2021)
- [Directed Acyclic Graph Network for Conversational Emotion Recognition](https://mlsanity.com/paper/2105.12907/) (2021)
- [Multi-attention Recurrent Network for Human Communication Comprehension](https://mlsanity.com/paper/1802.00923/) (2018)
- [DialogXL: All-in-One XLNet for Multi-Party Conversation Emotion Recognition](https://mlsanity.com/paper/2012.08695/) (2020)
- [DialogueCRN: Contextual Reasoning Networks for Emotion Recognition in Conversations](https://mlsanity.com/paper/2106.01978/) (2021)
- [DialogueTRM: Exploring the Intra- and Inter-Modal Emotional Behaviors in the Conversation](https://mlsanity.com/paper/2010.07637/) (2020)
- [SEOVER: Sentence-level Emotion Orientation Vector based Conversation Emotion Recognition Model](https://mlsanity.com/paper/2106.08785/) (2021)

## Built on by (most cited first, 30 of 48)

- [Optimus-1: Hybrid Multimodal Memory Empowered Agents Excel in Long-Horizon Tasks](https://mlsanity.com/paper/2408.03615/) (2024)
- [Deep Imbalanced Learning for Multimodal Emotion Recognition in Conversations](https://mlsanity.com/paper/2312.06337/) (2023)
- [Revisiting Multi-modal Emotion Learning with Broad State Space Models and Probability-guidance Fusion](https://mlsanity.com/paper/2404.17858/) (2024)
- [Revisiting Multimodal Emotion Recognition in Conversation from the Perspective of Graph Spectrum](https://mlsanity.com/paper/2404.17862/) (2024)
- [A Comprehensive Survey on Multi-modal Conversational Emotion Recognition with Deep Learning](https://mlsanity.com/paper/2312.05735/) (2023)
- [A Low-rank Matching Attention based Cross-modal Feature Fusion Method for Conversational Emotion Recognition](https://mlsanity.com/paper/2306.17799/) (2023)
- [DiaASQ : A Benchmark of Conversational Aspect-based Sentiment Quadruple Analysis](https://mlsanity.com/paper/2211.05705/) (2022)
- [UniSA: Unified Generative Framework for Sentiment Analysis](https://mlsanity.com/paper/2309.01339/) (2023)
- [Deep Emotion Recognition in Textual Conversations: A Survey](https://mlsanity.com/paper/2211.09172/) (2022)
- [DICNet: Deep Instance-Level Contrastive Network for Double Incomplete Multi-View Multi-Label Classification](https://mlsanity.com/paper/2303.08358/) (2023)
- [Recent Advances in Multimodal Affective Computing: An NLP Perspective](https://mlsanity.com/paper/2409.07388/) (2024)
- [A Two-Stage Multimodal Emotion Recognition Model Based on Graph Contrastive Learning](https://mlsanity.com/paper/2401.01495/) (2024)
- [TelME: Teacher-leading Multimodal Fusion Network for Emotion Recognition in Conversation](https://mlsanity.com/paper/2401.12987/) (2024)
- [HCAM -- Hierarchical Cross Attention Model for Multi-modal Emotion Recognition](https://mlsanity.com/paper/2304.06910/) (2023)
- [Multimodal Emotion Recognition in Conversations: A Survey of Methods, Trends, Challenges and Prospects](https://mlsanity.com/paper/2505.20511/) (2025)
- [MIPS at SemEval-2024 Task 3: Multimodal Emotion-Cause Pair Extraction in Conversations with Multimodal Language Models](https://mlsanity.com/paper/2404.00511/) (2024)
- [HCQA @ Ego4D EgoSchema Challenge 2024](https://mlsanity.com/paper/2406.15771/) (2024)
- [MSE-Adapter: A Lightweight Plugin Endowing LLMs with the Capability to Perform Multimodal Sentiment Analysis and Emotion Recognition](https://mlsanity.com/paper/2502.12478/) (2025)
- [UniMEEC: Towards Unified Multimodal Emotion Recognition and Emotion Cause](https://mlsanity.com/paper/2404.00403/) (2024)
- [Tracing Intricate Cues in Dialogue: Joint Graph Structure and Sentiment Dynamics for Multimodal Emotion Recognition](https://mlsanity.com/paper/2407.21536/) (2024)
- [EmotionIC: emotional inertia and contagion-driven dependency modeling for emotion recognition in conversation](https://mlsanity.com/paper/2303.11117/) (2023)
- [Multimodal Prompt Transformer with Hybrid Contrastive Learning for Emotion Recognition in Conversation](https://mlsanity.com/paper/2310.04456/) (2023)
- [From Abstraction to Instantiation: Learning Behavioral Representation for Vision-Language-Action Model](https://mlsanity.com/paper/2605.22671/) (2026)
- [Efficient Cross-Task Prompt Tuning for Few-Shot Conversational Emotion Recognition](https://mlsanity.com/paper/2310.14614/) (2023)
- [Unimodal-driven Distillation in Multimodal Emotion Recognition with Dynamic Fusion](https://mlsanity.com/paper/2503.23721/) (2025)
- [Navigating the Emotion Tree: Hierarchical Hyperbolic RAG for Multimodal Emotion Recognition](https://mlsanity.com/paper/2605.18884/) (2026)
- [Whose Emotion Matters? Speaking Activity Localisation without Prior Knowledge](https://mlsanity.com/paper/2211.15377/) (2022)
- [Emotion-Anchored Contrastive Learning Framework for Emotion Recognition in Conversation](https://mlsanity.com/paper/2403.20289/) (2024)
- [Dynamic Graph Neural ODE Network for Multi-modal Emotion Recognition in Conversation](https://mlsanity.com/paper/2412.02935/) (2024)
- [EmoVerse: Exploring Multimodal Large Language Models for Sentiment and Emotion Understanding](https://mlsanity.com/paper/2412.08049/) (2024)

## Abstract (arXiv)

Emotion recognition in conversation (ERC) aims to analyze the speaker's state and identify their emotion in the conversation. Recent works in ERC focus on context modeling but ignore the representation of contextual emotional tendency. In order to extract multi-modal information and the emotional tendency of the utterance effectively, we propose a new structure named Emoformer to extract multi-modal emotion vectors from different modalities and fuse them with sentence vector to be an emotion capsule. Furthermore, we design an end-to-end ERC model called EmoCaps, which extracts emotion vectors through the Emoformer structure and obtain the emotion classification results from a context analysis model. Through the experiments with two benchmark datasets, our model shows better performance than the existing state-of-the-art models.