mlsanityData to 7 Oct 2026

COGMEN: COntextualized GNN based Multimodal Emotion recognitioN

arXiv 2205.02455 ↗First posted 5 May 2022cs.CL cs.AI cs.LGcited by 29

Not on the map. Why?

Abstract (arXiv)

Emotions are an inherent part of human interactions, and consequently, it is imperative to develop AI systems that understand and recognize human emotions. During a conversation involving various people, a person's emotions are influenced by the other speaker's utterances and their own emotional state over the utterances. In this paper, we propose COntextualized Graph Neural Network based Multimodal Emotion recognitioN (COGMEN) system that leverages local information (i.e., inter/intra dependency between speakers) and global information (context). The proposed model uses Graph Neural Network (GNN) based architecture to model the complex dependencies (local and global information) in a conversation. Our model gives state-of-the-art (SOTA) results on IEMOCAP and MOSEI datasets, and detailed ablation experiments show the importance of modeling information at both levels.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Builds on 27 of its 62 references that are in the map. First posted on arXiv 5 May 2022.

Builds on

27 of 62 references.

References · 27

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  5. 2019Fast Graph Representation Learning with PyTorch Geometriccited by 2,414
  6. 2017Modeling Relational Data with Graph Convolutional Networkscited by 1,684
  7. 2020Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classificationcited by 500
  8. 2017Tensor Fusion Network for Multimodal Sentiment Analysiscited by 429
  9. 2019Deep Modular Co-Attention Networks for Visual Question Answeringcited by 338
  10. 2018DialogueRNN: An Attentive RNN for Emotion Detection in Conversationscited by 216
  11. 2019DialogueGCN: A Graph Convolutional Neural Network for Emotion Recognition in Conversationcited by 184
  12. 2018Memory Fusion Network for Multi-view Sequential Learningcited by 167
15 more
  1. 2019Emotion Recognition in Conversation: Research Challenges, Datasets, and Recent Advancescited by 118
  2. 2021Directed Acyclic Graph Network for Conversational Emotion Recognitioncited by 97
  3. 2020DialogXL: All-in-One XLNet for Multi-Party Conversation Emotion Recognitioncited by 78
  4. 2021DialogueCRN: Contextual Reasoning Networks for Emotion Recognition in Conversationscited by 71
  5. 2021EmoBERTa: Speaker-Aware Emotion Recognition in Conversation with RoBERTacited by 59
  6. 2019Affect-Driven Dialog Generationcited by 50
  7. 2020A Transformer-based joint-encoding for Emotion Recognition and Sentiment Analysiscited by 46
  8. 2018Multimodal Sentiment Analysis using Hierarchical Fusion with Context Modelingcited by 39
  9. 2020Multilogue-Net: A Context Aware RNN for Multi-modal Emotion Detection and Sentiment Analysis in Conversationcited by 27
  10. 2020Adapting a Language Model for Controlled Affective Text Generationcited by 22
  11. 2020Gated Mechanism for Attention Based Multimodal Sentiment Analysiscited by 17
  12. 2021Fine-Grained Emotion Prediction by Modeling Emotion Definitionscited by 9
  13. 2021Shapes of Emotions: Multimodal Emotion Recognition in Conversations via Emotion Shiftscited by 9
  14. 2019Domain Authoring Assistant for Intelligent Virtual Agentscited by 5
  15. 2021Developing emotion recognition for video conference software to support people with autismcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 8; 2024: 11; 2025: 3; 2026: 5.

29 papers.

Sort:

By area:

  1. 2022UniMSE: Towards Unified Multimodal Sentiment Analysis and Emotion Recognitioncited by 61
  2. 2023DER-GCN: Dialogue and Event Relation-Aware Graph Convolutional Neural Network for Multimodal Dialogue Emotion Recognitioncited by 23
  3. 2023Refashioning Emotion Recognition Modelling: The Advent of Generalised Large Modelscited by 16
  4. 2024Large Language Models Meet Text-Centric Multimodal Sentiment Analysis: A Surveycited by 16
  5. 2023CFN-ESA: A Cross-Modal Fusion Network with Emotion-Shift Awareness for Dialogue Emotion Recognitioncited by 14
  6. 2024Recent Advances in Multimodal Affective Computing: An NLP Perspectivecited by 12
  7. 2024Multimodal Machine Learning in Mental Health: A Survey of Data, Algorithms, and Challengescited by 7
  8. 2023Multimodal Prompt Transformer with Hybrid Contrastive Learning for Emotion Recognition in Conversationcited by 6
  9. 2024Multimodal Emotion Recognition using Audio-Video Transformer Fusion with Cross Attentioncited by 4
  10. 2024End-to-end Semantic-centric Video-based Multimodal Affective Computingcited by 4
  11. 2022Multi-Task Learning Framework for Extracting Emotion Cause Span and Entailment in Conversationscited by 3
  12. 2025DeepMLF: Multimodal language model with learnable tokens for deep fusion in sentiment analysiscited by 3
17 more
  1. 2023Towards Better Multi-modal Keyphrase Generation via Visual Entity Enhancement and Multi-granularity Image Noise Filteringcited by 2
  2. 2024Towards Data-centric Machine Learning on Directed Graphs: a Surveycited by 2
  3. 2024Effective Context Modeling Framework for Emotion Recognition in Conversationscited by 2
  4. 2023HIINT: Historical, Intra- and Inter- personal Dynamics Modeling with Cross-person Memory Transformercited by 1
  5. 2023A Contextualized Real-Time Multimodal Emotion Recognition for Conversational Agents using Graph Convolutional Networks in Reinforcement Learningcited by 1
  6. 2023PowMix: A Versatile Regularizer for Multimodal Sentiment Analysiscited by 1
  7. 2024Bridging Modalities: Knowledge Distillation and Masked Training for Translating Multi-Modal Emotion Recognition to Uni-Modal, Speech-Only Emotion Recognitioncited by 1
  8. 2024Push the Limit of Multi-modal Emotion Recognition by Prompting LLMs with Receptive-Field-Aware Attention Weightingcited by 0
  9. 2024WavFusion: Towards wav2vec 2.0 Multimodal Speech Emotion Recognitioncited by 0
  10. 2024Spatio-Temporal Fuzzy-oriented Multi-Modal Meta-Learning for Fine-grained Emotion Recognitioncited by 0
  11. 2025Multimodal Mixture of Low-Rank Experts for Sentiment Analysis and Emotion Recognitioncited by 0
  12. 2025Centering Emotion Hotspots: Multimodal Local-Global Fusion and Cross-Modal Alignment for Emotion Recognition in Conversationscited by 0
  13. 2026Divide and Refine: Enhancing Multimodal Representation and Explainability for Emotion Recognition in Conversationcited by 0
  14. 2026AMB-DSGDN: Adaptive Modality-Balanced Dynamic Semantic Graph Differential Network for Multimodal Emotion Recognitioncited by 0
  15. 2026Relational graph-driven differential denoising and diffusion attention fusion for multimodal conversation emotion recognitioncited by 0
  16. 2026ML-SAN: Multi-Level Speaker-Adaptive Network for Emotion Recognition in Conversationscited by 0
  17. 2026Leveraging Self-Paced Curriculum Learning for Enhanced Modality Balance in Multimodal Conversational Emotion Recognitioncited by 0