# GCNet: Graph Completion Network for Incomplete Multimodal Learning in Conversation

arXiv: https://arxiv.org/abs/2203.02177 · HTML: https://mlsanity.com/paper/2203.02177/

## Key facts

- Cited by 87 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 78 references that are in the map.
- First posted on arXiv 4 Mar 2022.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [Modeling Relational Data with Graph Convolutional Networks](https://mlsanity.com/paper/1703.06103/) (2017)
- [Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks](https://mlsanity.com/paper/1604.02878/) (2016)
- [Graph Convolutional Networks for Text Classification](https://mlsanity.com/paper/1809.05679/) (2018)
- [MISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2005.03545/) (2020)
- [SMIL: Multimodal Learning with Severely Missing Modality](https://mlsanity.com/paper/2103.05677/) (2021)
- [DialogueRNN: An Attentive RNN for Emotion Detection in Conversations](https://mlsanity.com/paper/1811.00405/) (2018)
- [DialogueGCN: A Graph Convolutional Neural Network for Emotion Recognition in Conversation](https://mlsanity.com/paper/1908.11540/) (2019)
- [On Deep Multi-View Representation Learning: Objectives and Optimization](https://mlsanity.com/paper/1602.01024/) (2016)
- [Found in Translation: Learning Robust Joint Representations by Cyclic Translations Between Modalities](https://mlsanity.com/paper/1812.07809/) (2018)
- [Emotion Recognition in Conversation: Research Challenges, Datasets, and Recent Advances](https://mlsanity.com/paper/1905.02947/) (2019)
- [Advances and Challenges in Conversational Recommender Systems: A Survey](https://mlsanity.com/paper/2101.09459/) (2021)
- [MMGCN: Multimodal Fusion via Deep Graph Convolution Network for Emotion Recognition in Conversation](https://mlsanity.com/paper/2107.06779/) (2021)
- [Exploring wav2vec 2.0 on speaker verification and language identification](https://mlsanity.com/paper/2012.06185/) (2020)
- [Learning to Extract Semantic Structure from Documents Using Multimodal Fully Convolutional Neural Network](https://mlsanity.com/paper/1706.02337/) (2017)
- [Learning Relationships between Text, Audio, and Video via Deep Canonical Correlation for Multimodal Language Analysis](https://mlsanity.com/paper/1911.05544/) (2019)
- [Deep Partial Multi-View Learning](https://mlsanity.com/paper/2011.06170/) (2020)
- [Variational Autoencoder with Arbitrary Conditioning](https://mlsanity.com/paper/1806.02382/) (2018)
- [Performance-Efficiency Trade-offs in Unsupervised Pre-training for Speech Recognition](https://mlsanity.com/paper/2109.06870/) (2021)
- [Modality to Modality Translation: An Adversarial Representation Learning and Graph Fusion Network for Multimodal Fusion](https://mlsanity.com/paper/1911.07848/) (2019)
- [Training Strategies to Handle Missing Modalities for Audio-Visual Expression Recognition](https://mlsanity.com/paper/2010.00734/) (2020)
- [Learning Representations from Imperfect Time Series Data via Tensor Rank Regularization](https://mlsanity.com/paper/1907.01011/) (2019)
- [Semi-supervised Deep Generative Modelling of Incomplete Multi-Modality Emotional Data](https://mlsanity.com/paper/1808.02096/) (2018)
- [Speech Emotion Recognition via Contrastive Loss under Siamese Networks](https://mlsanity.com/paper/1910.11174/) (2019)
- [Maximum Likelihood Estimation for Multimodal Learning with Missing Modality](https://mlsanity.com/paper/2108.10513/) (2021)

## Built on by (most cited first, 30 of 87)

- [Graph Foundation Models: Concepts, Opportunities and Challenges](https://mlsanity.com/paper/2310.11829/) (2023)
- [Deep Multimodal Learning with Missing Modality: A Survey](https://mlsanity.com/paper/2409.07825/) (2024)
- [Multimodal Representation Learning by Alternating Unimodal Adaptation](https://mlsanity.com/paper/2311.10707/) (2023)
- [Multimodal Fusion and Vision-Language Models: A Survey for Robot Vision](https://mlsanity.com/paper/2504.02477/) (2025)
- [MER 2023: Multi-label Learning, Modality Robustness, and Semi-Supervised Learning](https://mlsanity.com/paper/2304.08981/) (2023)
- [Revisiting Multimodal Emotion Recognition in Conversation from the Perspective of Graph Spectrum](https://mlsanity.com/paper/2404.17862/) (2024)
- [A Comprehensive Survey on Multi-modal Conversational Emotion Recognition with Deep Learning](https://mlsanity.com/paper/2312.05735/) (2023)
- [Correlation-Decoupled Knowledge Distillation for Multimodal Sentiment Analysis with Incomplete Modalities](https://mlsanity.com/paper/2404.16456/) (2024)
- [Coupled Mamba: Enhanced Multi-modal Fusion with Coupled State Space Model](https://mlsanity.com/paper/2405.18014/) (2024)
- [Graph Neural Networks for Financial Fraud Detection: A Review](https://mlsanity.com/paper/2411.05815/) (2024)
- [DecAlign: Hierarchical Cross-Modal Alignment for Decoupled Multimodal Representation Learning](https://mlsanity.com/paper/2503.11892/) (2025)
- [MER 2024: Semi-Supervised Learning, Noise Robustness, and Open-Vocabulary Multimodal Emotion Recognition](https://mlsanity.com/paper/2404.17113/) (2024)
- [Learning on Multimodal Graphs: A Survey](https://mlsanity.com/paper/2402.05322/) (2024)
- [Efficient Multimodal Transformer with Dual-Level Feature Restoration for Robust Multimodal Sentiment Analysis](https://mlsanity.com/paper/2208.07589/) (2022)
- [All in One Framework for Multimodal Re-identification in the Wild](https://mlsanity.com/paper/2405.04741/) (2024)
- [AlignMamba: Enhancing Multimodal Mamba with Local and Global Cross-modal Alignment](https://mlsanity.com/paper/2412.00833/) (2024)
- [Toward Robust Incomplete Multimodal Sentiment Analysis via Hierarchical Representation Learning](https://mlsanity.com/paper/2411.02793/) (2024)
- [SDR-GNN: Spectral Domain Reconstruction Graph Neural Network for Incomplete Multimodal Learning in Conversational Emotion Recognition](https://mlsanity.com/paper/2411.19822/) (2024)
- [Gradient-Guided Modality Decoupling for Missing-Modality Robustness](https://mlsanity.com/paper/2402.16318/) (2024)
- [Multimodal Emotion Recognition in Conversations: A Survey of Methods, Trends, Challenges and Prospects](https://mlsanity.com/paper/2505.20511/) (2025)
- [MuAP: Multi-step Adaptive Prompt Learning for Vision-Language Model with Missing Modality](https://mlsanity.com/paper/2409.04693/) (2024)
- [Knowledge Bridger: Towards Training-free Missing Modality Completion](https://mlsanity.com/paper/2502.19834/) (2025)
- [Tracing Intricate Cues in Dialogue: Joint Graph Structure and Sentiment Dynamics for Multimodal Emotion Recognition](https://mlsanity.com/paper/2407.21536/) (2024)
- [ReID5o: Achieving Omni Multi-modal Person Re-identification in a Single Model](https://mlsanity.com/paper/2506.09385/) (2025)
- [MCMoE: Completing Missing Modalities with Mixture of Experts for Incomplete Multimodal Action Quality Assessment](https://mlsanity.com/paper/2511.17397/) (2025)
- [Multimodal Prompt Transformer with Hybrid Contrastive Learning for Emotion Recognition in Conversation](https://mlsanity.com/paper/2310.04456/) (2023)
- [Missing-by-Design: Certifiable Modality Deletion for Revocable Multimodal Sentiment Analysis](https://mlsanity.com/paper/2602.16144/) (2026)
- [Spatial-Temporal Data Mining for Ocean Science: Data, Methodologies, and Opportunities](https://mlsanity.com/paper/2307.10803/) (2023)
- [End-to-end Semantic-centric Video-based Multimodal Affective Computing](https://mlsanity.com/paper/2408.07694/) (2024)
- [Towards Robust Multimodal Emotion Recognition under Missing Modalities and Distribution Shifts](https://mlsanity.com/paper/2506.10452/) (2025)

## Abstract (arXiv)

Conversations have become a critical data format on social media platforms. Understanding conversation from emotion, content and other aspects also attracts increasing attention from researchers due to its widespread application in human-computer interaction. In real-world environments, we often encounter the problem of incomplete modalities, which has become a core issue of conversation understanding. To address this problem, researchers propose various methods. However, existing approaches are mainly designed for individual utterances rather than conversational data, which cannot fully exploit temporal and speaker information in conversations. To this end, we propose a novel framework for incomplete multimodal learning in conversations, called "Graph Complete Network (GCNet)", filling the gap of existing works. Our GCNet contains two well-designed graph neural network-based modules, "Speaker GNN" and "Temporal GNN", to capture temporal and speaker dependencies. To make full use of complete and incomplete data, we jointly optimize classification and reconstruction tasks in an end-to-end manner. To verify the effectiveness of our method, we conduct experiments on three benchmark conversational datasets. Experimental results demonstrate that our GCNet is superior to existing state-of-the-art approaches in incomplete multimodal learning. Code is available at https://github.com/zeroQiaoba/GCNet.