# Correlation-Decoupled Knowledge Distillation for Multimodal Sentiment Analysis with Incomplete Modalities

arXiv: https://arxiv.org/abs/2404.16456 · HTML: https://mlsanity.com/paper/2404.16456/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal distillation, one of 12 directions in Knowledge distillation.
- Builds on 42 of its 70 references that are in the map.
- First posted on arXiv 25 Apr 2024.

## Placed in

- Knowledge distillation / [Multimodal distillation](https://mlsanity.com/distill/multimodal-distillation/)

## Builds on (most cited first, 30 of 42)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [Learning deep representations by mutual information estimation and maximization](https://mlsanity.com/paper/1808.06670/) (2018)
- [Relational Knowledge Distillation](https://mlsanity.com/paper/1904.05068/) (2019)
- [Contrastive Representation Distillation](https://mlsanity.com/paper/1910.10699/) (2019)
- [Improved Knowledge Distillation via Teacher Assistant](https://mlsanity.com/paper/1902.03393/) (2019)
- [Born Again Neural Networks](https://mlsanity.com/paper/1805.04770/) (2018)
- [Decoupled Knowledge Distillation](https://mlsanity.com/paper/2203.08679/) (2022)
- [On the Efficacy of Knowledge Distillation](https://mlsanity.com/paper/1910.01348/) (2019)
- [Variational Information Distillation for Knowledge Transfer](https://mlsanity.com/paper/1904.05835/) (2019)
- [Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlooks](https://mlsanity.com/paper/2004.05937/) (2020)
- [MISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2005.03545/) (2020)
- [Paraphrasing Complex Network: Network Compression via Factor Transfer](https://mlsanity.com/paper/1802.04977/) (2018)
- [SMIL: Multimodal Learning with Severely Missing Modality](https://mlsanity.com/paper/2103.05677/) (2021)
- [MOSI: Multimodal Corpus of Sentiment Intensity and Subjectivity Analysis in Online Opinion Videos](https://mlsanity.com/paper/1606.06259/) (2016)
- [Learning Modality-Specific Representations with Self-Supervised Multi-Task Learning for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2102.04830/) (2021)
- [Found in Translation: Learning Robust Joint Representations by Cyclic Translations Between Modalities](https://mlsanity.com/paper/1812.07809/) (2018)
- [Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2109.00412/) (2021)
- [Snapshot Distillation: Teacher-Student Optimization in One Generation](https://mlsanity.com/paper/1812.00123/) (2018)
- [Words Can Shift: Dynamically Adjusting Word Representations Using Nonverbal Behaviors](https://mlsanity.com/paper/1811.09362/) (2018)
- [GCNet: Graph Completion Network for Incomplete Multimodal Learning in Conversation](https://mlsanity.com/paper/2203.02177/) (2022)
- [Decoupled Multimodal Distilling for Emotion Recognition](https://mlsanity.com/paper/2303.13802/) (2023)
- [Knowledge distillation from multi-modal to mono-modal segmentation networks](https://mlsanity.com/paper/2106.09564/) (2021)
- [Spatio-Temporal Domain Awareness for Multi-Agent Collaborative Perception](https://mlsanity.com/paper/2307.13929/) (2023)
- [Learnable Cross-modal Knowledge Distillation for Multi-modal Learning with Missing Modality](https://mlsanity.com/paper/2310.01035/) (2023)
- [CubeMLP: An MLP-based Model for Multimodal Sentiment Analysis and Depression Estimation](https://mlsanity.com/paper/2207.14087/) (2022)
- [AIDE: A Vision-Driven Multi-View, Multi-Modal, Multi-Tasking Dataset for Assistive Driving Perception](https://mlsanity.com/paper/2307.13933/) (2023)
- [TSA-Net: Tube Self-Attention Network for Action Quality Assessment](https://mlsanity.com/paper/2201.03746/) (2022)

## Built on by (most cited first, 30 of 32)

- [Deep Multimodal Learning with Missing Modality: A Survey](https://mlsanity.com/paper/2409.07825/) (2024)
- [DLF: Disentangled-Language-Focused Multimodal Sentiment Analysis](https://mlsanity.com/paper/2412.12225/) (2024)
- [Toward Robust Incomplete Multimodal Sentiment Analysis via Hierarchical Representation Learning](https://mlsanity.com/paper/2411.02793/) (2024)
- [Retrieval-Augmented Dynamic Prompt Tuning for Incomplete Multimodal Learning](https://mlsanity.com/paper/2501.01120/) (2025)
- [Multi-modal Knowledge Distillation-based Human Trajectory Forecasting](https://mlsanity.com/paper/2503.22201/) (2025)
- [Test-Time Distillation for Continual Model Adaptation](https://mlsanity.com/paper/2506.02671/) (2025)
- [Enhancing Multimodal Sentiment Analysis for Missing Modality through Self-Distillation and Unified Modality Cross-Attention](https://mlsanity.com/paper/2410.15029/) (2024)
- [Multimodal Online Federated Learning with Modality Missing in Internet of Things](https://mlsanity.com/paper/2505.16138/) (2025)
- [ICYM2I: The illusion of multimodal informativeness under missingness](https://mlsanity.com/paper/2505.16953/) (2025)
- [Hyper-modal Imputation Diffusion Embedding with Dual-Distillation for Federated Multimodal Knowledge Graph Completion](https://mlsanity.com/paper/2506.22036/) (2025)
- [ModalImmune: Immunity Driven Unlearning via Self Destructive Training](https://mlsanity.com/paper/2602.16197/) (2026)
- [Addressing Missing and Noisy Modalities in One Solution: Unified Modality-Quality Framework for Low-quality Multimodal Data](https://mlsanity.com/paper/2603.02695/) (2026)
- [Progressive Representation Learning for Multimodal Sentiment Analysis with Incomplete Modalities](https://mlsanity.com/paper/2603.09111/) (2026)
- [BALM: A Model-Agnostic Framework for Balanced Multimodal Learning under Imbalanced Missing Rates](https://mlsanity.com/paper/2603.19718/) (2026)
- [Enhance-then-Balance Modality Collaboration for Robust Multimodal Sentiment Analysis](https://mlsanity.com/paper/2604.12518/) (2026)
- [Modality Disentangled Learning for Incomplete Multimodal Emotion Recognition: A Primitive Memory Distillation Perspective](https://mlsanity.com/paper/2608.30563/) (2026)
- [Are you SURE? Enhancing Multimodal Pretraining with Missing Modalities through Uncertainty Estimation](https://mlsanity.com/paper/2504.13465/) (2025)
- [Confidence-Aware Self-Distillation for Multimodal Sentiment Analysis with Incomplete Modalities](https://mlsanity.com/paper/2506.01490/) (2025)
- [Toward Robust Semi-supervised Regression via Dual-stream Knowledge Distillation](https://mlsanity.com/paper/2508.14082/) (2025)
- [Information-Theoretic Criteria for Knowledge Distillation in Multimodal Learning](https://mlsanity.com/paper/2510.13182/) (2025)
- [FSRF: Factorization-guided Semantic Recovery for Incomplete Multimodal Sentiment Analysis](https://mlsanity.com/paper/2510.16086/) (2025)
- [OMG-Agent: Toward Robust Missing Modality Generation with Decoupled Coarse-to-Fine Agentic Workflows](https://mlsanity.com/paper/2602.04144/) (2026)
- [MuDD: A Multimodal Deception Detection Dataset and GSR-Guided Progressive Distillation for Non-Contact Deception Detection](https://mlsanity.com/paper/2603.26064/) (2026)
- [Disentangled Fine-Grained Prototype Learning for Incomplete Image-Tabular Classification](https://mlsanity.com/paper/2606.05455/) (2026)
- [When Does Quality-Aware Multimodal Fusion Matter? A Leakage-Safe Diagnostic for Decision-Level Dependence](https://mlsanity.com/paper/2606.26473/) (2026)
- [Should Missing Modalities Always Be Necessary to Repair for Multi-modal Sentiment Analysis?](https://mlsanity.com/paper/2607.17262/) (2026)
- [Rethinking Modality Reliability in Multimodal Sentiment Analysis with Incomplete Observations](https://mlsanity.com/paper/2608.03611/) (2026)
- [MIDAS: Mutual Information Disentanglement with Uncertainty-Aware Fusion for Incomplete Multimodal Sentiment Analysis](https://mlsanity.com/paper/2608.09986/) (2026)
- [Multi-Modal Anomaly Detection: A Survey](https://mlsanity.com/paper/2608.24937/) (2026)
- [Vision-Guided Text Prompt Tuning for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2609.06497/) (2026)

## Abstract (arXiv)

Multimodal sentiment analysis (MSA) aims to understand human sentiment through multimodal data. Most MSA efforts are based on the assumption of modality completeness. However, in real-world applications, some practical factors cause uncertain modality missingness, which drastically degrades the model's performance. To this end, we propose a Correlation-decoupled Knowledge Distillation (CorrKD) framework for the MSA task under uncertain missing modalities. Specifically, we present a sample-level contrastive distillation mechanism that transfers comprehensive knowledge containing cross-sample correlations to reconstruct missing semantics. Moreover, a category-guided prototype distillation mechanism is introduced to capture cross-category correlations using category prototypes to align feature distributions and generate favorable joint representations. Eventually, we design a response-disentangled consistency distillation strategy to optimize the sentiment decision boundaries of the student network through response disentanglement and mutual information maximization. Comprehensive experiments on three datasets indicate that our framework can achieve favorable improvements compared with several baselines.