# MMGCN: Multimodal Fusion via Deep Graph Convolution Network for Emotion Recognition in Conversation

arXiv: https://arxiv.org/abs/2107.06779 · HTML: https://mlsanity.com/paper/2107.06779/

## Key facts

- Cited by 99 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 26 references that are in the map.
- First posted on arXiv 14 Jul 2021.

## Builds on (most cited first, 17 of 17)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks](https://mlsanity.com/paper/1810.02244/) (2018)
- [Simple and Deep Graph Convolutional Networks](https://mlsanity.com/paper/2007.02133/) (2020)
- [Gated Graph Sequence Neural Networks](https://mlsanity.com/paper/1511.05493/) (2015)
- [Multimodal Transformer for Unaligned Multimodal Language Sequences](https://mlsanity.com/paper/1906.00295/) (2019)
- [MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations](https://mlsanity.com/paper/1810.02508/) (2018)
- [DeepGCNs: Can GCNs Go as Deep as CNNs?](https://mlsanity.com/paper/1904.03751/) (2019)
- [DialogueRNN: An Attentive RNN for Emotion Detection in Conversations](https://mlsanity.com/paper/1811.00405/) (2018)
- [DialogueGCN: A Graph Convolutional Neural Network for Emotion Recognition in Conversation](https://mlsanity.com/paper/1908.11540/) (2019)
- [Memory Fusion Network for Multi-view Sequential Learning](https://mlsanity.com/paper/1802.00927/) (2018)
- [Training Deep Networks for Facial Expression Recognition with Crowd-Sourced Label Distribution](https://mlsanity.com/paper/1608.01041/) (2016)
- [EmotionLines: An Emotion Corpus of Multi-Party Conversations](https://mlsanity.com/paper/1802.08379/) (2018)
- [Multimodal Sentiment Analysis with Word-Level Fusion and Reinforcement Learning](https://mlsanity.com/paper/1802.00924/) (2018)
- [Multimodal Relational Tensor Network for Sentiment and Emotion Classification](https://mlsanity.com/paper/1806.02923/) (2018)

## Built on by (most cited first, 30 of 99)

- [GCNet: Graph Completion Network for Incomplete Multimodal Learning in Conversation](https://mlsanity.com/paper/2203.02177/) (2022)
- [Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Survey](https://mlsanity.com/paper/2411.17558/) (2024)
- [UniMSE: Towards Unified Multimodal Sentiment Analysis and Emotion Recognition](https://mlsanity.com/paper/2211.11256/) (2022)
- [MM-DFN: Multimodal Dynamic Fusion Network for Emotion Recognition in Conversations](https://mlsanity.com/paper/2203.02385/) (2022)
- [M3ED: Multi-modal Multi-scene Multi-label Emotional Dialogue Database](https://mlsanity.com/paper/2205.10237/) (2022)
- [EmoCaps: Emotion Capsule based Model for Conversational Emotion Recognition](https://mlsanity.com/paper/2203.13504/) (2022)
- [Deep Imbalanced Learning for Multimodal Emotion Recognition in Conversations](https://mlsanity.com/paper/2312.06337/) (2023)
- [A Transformer-Based Model With Self-Distillation for Multimodal Emotion Recognition in Conversations](https://mlsanity.com/paper/2310.20494/) (2023)
- [Revisiting Multi-modal Emotion Learning with Broad State Space Models and Probability-guidance Fusion](https://mlsanity.com/paper/2404.17858/) (2024)
- [Revisiting Multimodal Emotion Recognition in Conversation from the Perspective of Graph Spectrum](https://mlsanity.com/paper/2404.17862/) (2024)
- [Multimodal Emotion-Cause Pair Extraction in Conversations](https://mlsanity.com/paper/2110.08020/) (2021)
- [A Low-rank Matching Attention based Cross-modal Feature Fusion Method for Conversational Emotion Recognition](https://mlsanity.com/paper/2306.17799/) (2023)
- [MIntRec2.0: A Large-scale Benchmark Dataset for Multimodal Intent Recognition and Out-of-scope Detection in Conversations](https://mlsanity.com/paper/2403.10943/) (2024)
- [GraphCFC: A Directed Graph Based Cross-Modal Feature Complementation Approach for Multimodal Conversational Emotion Recognition](https://mlsanity.com/paper/2207.12261/) (2022)
- [Supervised Adversarial Contrastive Learning for Emotion Recognition in Conversations](https://mlsanity.com/paper/2306.01505/) (2023)
- [Masked Graph Learning with Recurrent Alignment for Multimodal Emotion Recognition in Conversation](https://mlsanity.com/paper/2407.16714/) (2024)
- [UniSA: Unified Generative Framework for Sentiment Analysis](https://mlsanity.com/paper/2309.01339/) (2023)
- [Dyadformer: A Multi-modal Transformer for Long-Range Modeling of Dyadic Interactions](https://mlsanity.com/paper/2109.09487/) (2021)
- [Large Language Models Meet Text-Centric Multimodal Sentiment Analysis: A Survey](https://mlsanity.com/paper/2406.08068/) (2024)
- [CFN-ESA: A Cross-Modal Fusion Network with Emotion-Shift Awareness for Dialogue Emotion Recognition](https://mlsanity.com/paper/2307.15432/) (2023)
- [Deep Emotion Recognition in Textual Conversations: A Survey](https://mlsanity.com/paper/2211.09172/) (2022)
- [Revisiting Disentanglement and Fusion on Modality and Context in Conversational Multimodal Emotion Recognition](https://mlsanity.com/paper/2308.04502/) (2023)
- [Recent Advances in Multimodal Affective Computing: An NLP Perspective](https://mlsanity.com/paper/2409.07388/) (2024)
- [SDR-GNN: Spectral Domain Reconstruction Graph Neural Network for Incomplete Multimodal Learning in Conversational Emotion Recognition](https://mlsanity.com/paper/2411.19822/) (2024)
- [Adversarial Representation with Intra-Modal and Inter-Modal Graph Contrastive Learning for Multimodal Emotion Recognition](https://mlsanity.com/paper/2312.16778/) (2023)
- [A Two-Stage Multimodal Emotion Recognition Model Based on Graph Contrastive Learning](https://mlsanity.com/paper/2401.01495/) (2024)
- [TelME: Teacher-leading Multimodal Fusion Network for Emotion Recognition in Conversation](https://mlsanity.com/paper/2401.12987/) (2024)
- [Multimodal Emotion Recognition in Conversations: A Survey of Methods, Trends, Challenges and Prospects](https://mlsanity.com/paper/2505.20511/) (2025)
- [A Quantum Probability Driven Framework for Joint Multi-Modal Sarcasm, Sentiment and Emotion Analysis](https://mlsanity.com/paper/2306.03650/) (2023)
- [EmoDynamiX: Emotional Support Dialogue Strategy Prediction by Modelling MiXed Emotions and Discourse Dynamics](https://mlsanity.com/paper/2408.08782/) (2024)

## Abstract (arXiv)

Emotion recognition in conversation (ERC) is a crucial component in affective dialogue systems, which helps the system understand users' emotions and generate empathetic responses. However, most works focus on modeling speaker and contextual information primarily on the textual modality or simply leveraging multimodal information through feature concatenation. In order to explore a more effective way of utilizing both multimodal and long-distance contextual information, we propose a new model based on multimodal fused graph convolutional network, MMGCN, in this work. MMGCN can not only make use of multimodal dependencies effectively, but also leverage speaker information to model inter-speaker and intra-speaker dependency. We evaluate our proposed model on two public benchmark datasets, IEMOCAP and MELD, and the results prove the effectiveness of MMGCN, which outperforms other SOTA methods by a significant margin under the multimodal conversation setting.