mlsanityData to 7 Oct 2026

Decoupled Multimodal Distilling for Emotion Recognition

arXiv 2303.13802 ↗First posted 24 Mar 2023cs.CVcited by 76

Not on the map. Why?

Abstract (arXiv)

Human multimodal emotion recognition (MER) aims to perceive human emotions via language, visual and acoustic modalities. Despite the impressive performance of previous MER approaches, the inherent multimodal heterogeneities still haunt and the contribution of different modalities varies significantly. In this work, we mitigate this issue by proposing a decoupled multimodal distillation (DMD) approach that facilitates flexible and adaptive crossmodal knowledge distillation, aiming to enhance the discriminative features of each modality. Specially, the representation of each modality is decoupled into two parts, i.e., modality-irrelevant/-exclusive spaces, in a self-regression manner. DMD utilizes a graph distillation unit (GD-Unit) for each decoupled part so that each GD can be performed in a more specialized and effective manner. A GD-Unit consists of a dynamic graph where each vertice represents a modality and each edge indicates a dynamic knowledge distillation. Such GD paradigm provides a flexible knowledge transfer manner where the distillation weights can be automatically learned, thus enabling diverse crossmodal knowledge transfer patterns. Experimental results show DMD consistently obtains superior performance than state-of-the-art MER methods. Visualization results show the graph edges in DMD exhibit meaningful distributional patterns w.r.t. the modality-irrelevant/-exclusive feature spaces. Codes are released at urlhttps://github.com/mdswyz/DMD.

Key facts. Cited by 76 papers in the map (data through 7 Oct 2026). Builds on 20 of its 39 references that are in the map. First posted on arXiv 24 Mar 2023.

Builds on

20 of 39 references.

References · 20

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  3. 2014FitNets: Hints for Thin Deep Netscited by 1,619
  4. 2019Relational Knowledge Distillationcited by 873
  5. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 718
  6. 2019Multimodal Transformer for Unaligned Multimodal Language Sequencescited by 637
  7. 2019Improved Knowledge Distillation via Teacher Assistantcited by 567
  8. 2018Born Again Neural Networkscited by 539
  9. 2017Tensor Fusion Network for Multimodal Sentiment Analysiscited by 429
  10. 2022Decoupled Knowledge Distillationcited by 380
  11. 2019A Comprehensive Overhaul of Feature Distillationcited by 341
  12. 2018Efficient Low-rank Multimodal Fusion with Modality-Specific Factorscited by 268
8 more
  1. 2020MISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysiscited by 262
  2. 2018Learning Factorized Multimodal Representationscited by 171
  3. 2018Memory Fusion Network for Multi-view Sequential Learningcited by 167
  4. 2018Found in Translation: Learning Robust Joint Representations by Cyclic Translations Between Modalitiescited by 149
  5. 2018Words Can Shift: Dynamically Adjusting Word Representations Using Nonverbal Behaviorscited by 89
  6. 2017Graph Distillation for Action Detection with Privileged Modalitiescited by 34
  7. 2018Better and Faster: Knowledge Transfer from Multiple Self-supervised Learning Tasks via Graph Distillation for Video Classificationcited by 20
  8. 2019Knowledge Transfer Graph for Deep Collaborative Learningcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 22; 2025: 21; 2026: 31.

76 papers.

Sort:

By area:

  1. 2024Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction Tuningcited by 117
  2. 2024Correlation-Decoupled Knowledge Distillation for Multimodal Sentiment Analysis with Incomplete Modalitiescited by 32
  3. 2024Coupled Mamba: Enhanced Multi-modal Fusion with Coupled State Space Modelcited by 30
  4. 2024Towards Multimodal Sentiment Analysis Debiasing via Bias Purificationcited by 26
  5. 2024Facial Affective Behavior Analysis with Instruction Tuningcited by 26
  6. 2025DecAlign: Hierarchical Cross-Modal Alignment for Decoupled Multimodal Representation Learningcited by 23
  7. 2024AlignMamba: Enhancing Multimodal Mamba with Local and Global Cross-modal Alignmentcited by 17
  8. 2024Toward Robust Incomplete Multimodal Sentiment Analysis via Hierarchical Representation Learningcited by 16
  9. 2025DynCIM: Dynamic Curriculum for Imbalanced Multimodal Learningcited by 16
  10. 2024Recursive Joint Cross-Modal Attention for Multimodal Fusion in Dimensional Emotion Recognitioncited by 15
  11. 2024Cohort-Individual Cooperative Learning for Multimodal Cancer Survival Analysiscited by 13
  12. 2024Recent Advances in Multimodal Affective Computing: An NLP Perspectivecited by 12
38 more
  1. 2024Asynchronous Multimodal Video Sequence Fusion via Learning Modality-Exclusive and -Agnostic Representationscited by 10
  2. 2025Multi-modal Knowledge Distillation-based Human Trajectory Forecastingcited by 10
  3. 2025Towards Robust Multimodal Physiological Foundation Models: Handling Arbitrary Missing Modalitiescited by 10
  4. 2023MASA-TCN: Multi-anchor Space-aware Temporal Convolutional Neural Networks for Continuous and Discrete EEG Emotion Recognitioncited by 9
  5. 2025Modality-Invariant Bidirectional Temporal Representation Distillation Network for Missing Multimodal Sentiment Analysiscited by 7
  6. 2026Unbiased Dynamic Multimodal Fusioncited by 7
  7. 2024Efficient Feature Extraction and Late Fusion Strategy for Audiovisual Emotional Mimicry Intensity Estimationcited by 6
  8. 2026Emotion-LLaMAv2 and MMEVerse: A New Framework and Benchmark for Multimodal Emotion Understandingcited by 5
  9. 2026Tri-Subspaces Disentanglement for Multimodal Sentiment Analysiscited by 5
  10. 2026CLCR: Cross-Level Semantic Collaborative Representation for Multimodal Learningcited by 5
  11. 2024Triple Disentangled Representation Learning for Multimodal Affective Analysiscited by 4
  12. 2024Completed Feature Disentanglement Learning for Multimodal MRIs Analysiscited by 4
  13. 2024Resolving Sentiment Discrepancy for Multimodal Sentiment Detection via Semantics Completion and Decompositioncited by 4
  14. 2025Towards Robust Multimodal Emotion Recognition under Missing Modalities and Distribution Shiftscited by 4
  15. 2025TiCAL:Typicality-Based Consistency-Aware Learning for Multimodal Emotion Recognitioncited by 4
  16. 2024EmoVerse: Exploring Multimodal Large Language Models for Sentiment and Emotion Understandingcited by 3
  17. 2026Temporal-Spatial Decouple before Act: Disentangled Representation Learning for Multimodal Sentiment Analysiscited by 3
  18. 2025Disentangling Bias by Modeling Intra- and Inter-modal Causal Attention for Multimodal Sentiment Analysiscited by 2
  19. 2025Towards Minimal Causal Representations for Human Multimodal Language Understandingcited by 2
  20. 2026Complementarity-Supervised Spectral-Band Routing for Multimodal Emotion Recognitioncited by 2
  21. 2023Balanced Multi-modal Federated Learning via Cross-Modal Infiltrationcited by 1
  22. 2024Cooperative Sentiment Agents for Multimodal Sentiment Analysiscited by 1
  23. 2024Adaptive Modality Balanced Online Knowledge Distillation for Brain-Eye-Computer based Dim Object Detectioncited by 1
  24. 2024Non-target Divergence Hypothesis: Toward Understanding Domain Gaps in Cross-Modal Knowledge Distillationcited by 1
  25. 2024MVKTrans: Multi-View Knowledge Transfer for Robust Multiomics Classificationcited by 1
  26. 2025Leveraging CLIP Encoder for Multimodal Emotion Recognitioncited by 1
  27. 2025Calibrating Multimodal Consensus for Emotion Recognitioncited by 1
  28. 2025DashFusion: Dual-stream Alignment with Hierarchical Bottleneck Fusion for Multimodal Sentiment Analysiscited by 1
  29. 2026Decoupled Hierarchical Distillation for Multimodal Emotion Recognitioncited by 1
  30. 2026Enhancing Weakly Supervised Multimodal Video Anomaly Detection through Text Guidancecited by 1
  31. 2026CaReFlow: Cyclic Adaptive Rectified Flow for Multimodal Fusioncited by 1
  32. 2026Memory-guided Prototypical Co-occurrence Learning for Mixed Emotion Recognitioncited by 1
  33. 2026Addressing Missing and Noisy Modalities in One Solution: Unified Modality-Quality Framework for Low-quality Multimodal Datacited by 1
  34. 2026CFCML: A Coarse-to-Fine Crossmodal Learning Framework For Disease Diagnosis Using Multimodal Images and Tabular Datacited by 1
  35. 2026Enhance-then-Balance Modality Collaboration for Robust Multimodal Sentiment Analysiscited by 1
  36. 2026Controlling Decision Drift in Multimodal Sentiment Analysis with Missing Modalitiescited by 1
  37. 2024TCAN: Text-oriented Cross Attention Network for Multimodal Sentiment Analysiscited by 0
  38. 2024Seeking the Sufficiency and Necessity Causal Features in Multimodal Representation Learningcited by 0

50 most cited of 76 All 76 (JSON)