mlsanityData to 14 Oct 2026

Correlation-Decoupled Knowledge Distillation for Multimodal Sentiment Analysis with Incomplete Modalities

arXiv 2404.16456 ↗First posted 25 Apr 2024cs.CVcited by 32

On the map: Knowledge distillation → Multimodal distillation

Abstract (arXiv)

Multimodal sentiment analysis (MSA) aims to understand human sentiment through multimodal data. Most MSA efforts are based on the assumption of modality completeness. However, in real-world applications, some practical factors cause uncertain modality missingness, which drastically degrades the model's performance. To this end, we propose a Correlation-decoupled Knowledge Distillation (CorrKD) framework for the MSA task under uncertain missing modalities. Specifically, we present a sample-level contrastive distillation mechanism that transfers comprehensive knowledge containing cross-sample correlations to reconstruct missing semantics. Moreover, a category-guided prototype distillation mechanism is introduced to capture cross-category correlations using category prototypes to align feature distributions and generate favorable joint representations. Eventually, we design a response-disentangled consistency distillation strategy to optimize the sentiment decision boundaries of the student network through response disentanglement and mutual information maximization. Comprehensive experiments on three datasets indicate that our framework can achieve favorable improvements compared with several baselines.

Key facts. Cited by 32 papers in the map (data through 14 Oct 2026). Placed in Multimodal distillation, one of 12 directions in Knowledge distillation. Builds on 42 of its 70 references that are in the map. First posted on arXiv 25 Apr 2024.

Builds on

42 of 70 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023Learnable Cross-modal Knowledge Distillation for Multi-modal Learning with Missing Modality2 Oct
  2. 2023Robust Cross-Modal Knowledge Distillation for Unconstrained Videos16 Apr
  3. 2023Decoupled Multimodal Distilling for Emotion Recognition24 Mar

Other references · 39

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  3. 2017Prototypical Networks for Few-shot Learningcited by 3,015
  4. 2016Pyramid Scene Parsing Networkcited by 2,479
  5. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,273
  6. 2019Relational Knowledge Distillationcited by 889
  7. 2019Contrastive Representation Distillationcited by 748
  8. 2019Improved Knowledge Distillation via Teacher Assistantcited by 573
  9. 2018Born Again Neural Networkscited by 546
  10. 2022Decoupled Knowledge Distillationcited by 388
  11. 2019On the Efficacy of Knowledge Distillationcited by 381
  12. 2019Variational Information Distillation for Knowledge Transfercited by 324
27 more
  1. 2020Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlookscited by 316
  2. 2020MISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysiscited by 265
  3. 2018Paraphrasing Complex Network: Network Compression via Factor Transfercited by 260
  4. 2021SMIL: Multimodal Learning with Severely Missing Modalitycited by 245
  5. 2016MOSI: Multimodal Corpus of Sentiment Intensity and Subjectivity Analysis in Online Opinion Videoscited by 194
  6. 2021Learning Modality-Specific Representations with Self-Supervised Multi-Task Learning for Multimodal Sentiment Analysiscited by 180
  7. 2018Found in Translation: Learning Robust Joint Representations by Cyclic Translations Between Modalitiescited by 149
  8. 2021Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysiscited by 131
  9. 2018Snapshot Distillation: Teacher-Student Optimization in One Generationcited by 99
  10. 2018Words Can Shift: Dynamically Adjusting Word Representations Using Nonverbal Behaviorscited by 89
  11. 2022GCNet: Graph Completion Network for Incomplete Multimodal Learning in Conversationcited by 87
  12. 2021Knowledge distillation from multi-modal to mono-modal segmentation networkscited by 68
  13. 2023Spatio-Temporal Domain Awareness for Multi-Agent Collaborative Perceptioncited by 52
  14. 2022CubeMLP: An MLP-based Model for Multimodal Sentiment Analysis and Depression Estimationcited by 40
  15. 2023AIDE: A Vision-Driven Multi-View, Multi-Modal, Multi-Tasking Dataset for Assistive Driving Perceptioncited by 39
  16. 2022TSA-Net: Tube Self-Attention Network for Action Quality Assessmentcited by 36
  17. 2023Context De-confounded Emotion Recognitioncited by 34
  18. 2018Semi-supervised Deep Generative Modelling of Incomplete Multi-Modality Emotional Datacited by 29
  19. 2022Tag-assisted Multimodal Sentiment Analysis under Uncertain Missing Modalitiescited by 28
  20. 2024Towards Multimodal Sentiment Analysis Debiasing via Bias Purificationcited by 27
  21. 2024Robust Emotion Recognition in Context Debiasingcited by 24
  22. 2024MISS: A Generative Pretraining and Finetuning Approach for Med-VQAcited by 14
  23. 2023Text-oriented Modality Reinforcement Network for Multimodal Sentiment Analysis from Unaligned Multimodal Sequencescited by 13
  24. 2024Debiased Multimodal Understanding for Human Language Sequencescited by 10
  25. 2023Towards Simultaneous Segmentation of Liver Tumors and Intrahepatic Vessels via Cross-attention Mechanismcited by 8
  26. 2019Online Sensor Hallucination via Knowledge Distillation for Multimodal Image Classificationcited by 3
  27. 2021QuTI! Quantifying Text-Image Consistency in Multimodal Documentscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 11; 2026: 17.

32 papers.

Sort:

By area:

  1. 2024Deep Multimodal Learning with Missing Modality: A Surveycited by 92
  2. 2024DLF: Disentangled-Language-Focused Multimodal Sentiment Analysiscited by 27
  3. 2024Toward Robust Incomplete Multimodal Sentiment Analysis via Hierarchical Representation Learningcited by 16
  4. 2025Retrieval-Augmented Dynamic Prompt Tuning for Incomplete Multimodal Learningcited by 15
  5. 2025Multi-modal Knowledge Distillation-based Human Trajectory Forecastingcited by 10
  6. 2025Test-Time Distillation for Continual Model Adaptationcited by 5
  7. 2024Enhancing Multimodal Sentiment Analysis for Missing Modality through Self-Distillation and Unified Modality Cross-Attentioncited by 2
  8. 2025Multimodal Online Federated Learning with Modality Missing in Internet of Thingscited by 2
  9. 2025ICYM2I: The illusion of multimodal informativeness under missingnesscited by 1
  10. 2025Hyper-modal Imputation Diffusion Embedding with Dual-Distillation for Federated Multimodal Knowledge Graph Completioncited by 1
  11. 2026ModalImmune: Immunity Driven Unlearning via Self Destructive Trainingcited by 1
  12. 2026Addressing Missing and Noisy Modalities in One Solution: Unified Modality-Quality Framework for Low-quality Multimodal Datacited by 1
20 more
  1. 2026Progressive Representation Learning for Multimodal Sentiment Analysis with Incomplete Modalitiescited by 1
  2. 2026BALM: A Model-Agnostic Framework for Balanced Multimodal Learning under Imbalanced Missing Ratescited by 1
  3. 2026Enhance-then-Balance Modality Collaboration for Robust Multimodal Sentiment Analysiscited by 1
  4. 2026Modality Disentangled Learning for Incomplete Multimodal Emotion Recognition: A Primitive Memory Distillation Perspectivecited by 1
  5. 2025Are you SURE? Enhancing Multimodal Pretraining with Missing Modalities through Uncertainty Estimationcited by 0
  6. 2025Confidence-Aware Self-Distillation for Multimodal Sentiment Analysis with Incomplete Modalitiescited by 0
  7. 2025Toward Robust Semi-supervised Regression via Dual-stream Knowledge Distillationcited by 0
  8. 2025Information-Theoretic Criteria for Knowledge Distillation in Multimodal Learningcited by 0
  9. 2025FSRF: Factorization-guided Semantic Recovery for Incomplete Multimodal Sentiment Analysiscited by 0
  10. 2026OMG-Agent: Toward Robust Missing Modality Generation with Decoupled Coarse-to-Fine Agentic Workflowscited by 0
  11. 2026MuDD: A Multimodal Deception Detection Dataset and GSR-Guided Progressive Distillation for Non-Contact Deception Detectioncited by 0
  12. 2026Disentangled Fine-Grained Prototype Learning for Incomplete Image-Tabular Classificationcited by 0
  13. 2026When Does Quality-Aware Multimodal Fusion Matter? A Leakage-Safe Diagnostic for Decision-Level Dependencecited by 0
  14. 2026Should Missing Modalities Always Be Necessary to Repair for Multi-modal Sentiment Analysis?cited by 0
  15. 2026Rethinking Modality Reliability in Multimodal Sentiment Analysis with Incomplete Observationscited by 0
  16. 2026MIDAS: Mutual Information Disentanglement with Uncertainty-Aware Fusion for Incomplete Multimodal Sentiment Analysiscited by 0
  17. 2026Multi-Modal Anomaly Detection: A Surveycited by 0
  18. 2026Vision-Guided Text Prompt Tuning for Multimodal Sentiment Analysiscited by 0
  19. 2026SemMSA: Latent Semantic-Aided Robust Multimodal Sentiment Analysis with Incomplete Datacited by 0
  20. 2026Reliability-aware Cross-sample Enhancement for Robust Multimodal Sentiment Analysiscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Modality-Invariant Bidirectional Temporal Representation Distillation Network for Missing Multimodal Sentiment Analysis
  2. 2025Towards Explainable Fusion and Balanced Learning in Multimodal Sentiment Analysis
  3. 2026Decoupled Hierarchical Distillation for Multimodal Emotion Recognition
  4. 2025Disentangling Bias by Modeling Intra- and Inter-modal Causal Attention for Multimodal Sentiment Analysis
  5. 2026Robust Incomplete Multimodal Sentiment Analysis via Iterative Proxy Correction
  6. 2024Multimodal Sentiment Analysis Based on Causal Reasoning
  7. 2025Multi-Modality Collaborative Learning for Sentiment Analysis
  8. 2025Resource-Limited Joint Multimodal Sentiment Reasoning and Classification via Chain-of-Thought Enhancement and Distillation