# Fine-grained Disentangled Representation Learning for Multimodal Emotion Recognition

arXiv: https://arxiv.org/abs/2312.13567 · HTML: https://mlsanity.com/paper/2312.13567/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Dec 2023.

## Built on by (most cited first, 11 of 11)

- [Serialized Speech Information Guidance with Overlapped Encoding Separation for Multi-Speaker Automatic Speech Recognition](https://mlsanity.com/paper/2409.00815/) (2024)
- [Adapting Whisper for Code-Switching through Encoding Refining and Language-Aware Decoding](https://mlsanity.com/paper/2412.16507/) (2024)
- [Decoupled Audio-Visual Dataset Distillation](https://mlsanity.com/paper/2511.17890/) (2025)
- [Tri-Subspaces Disentanglement for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2602.19585/) (2026)
- [Enhancing Multimodal Emotion Recognition through Multi-Granularity Cross-Modal Alignment](https://mlsanity.com/paper/2412.20821/) (2024)
- [CLCR: Cross-Level Semantic Collaborative Representation for Multimodal Learning](https://mlsanity.com/paper/2602.19605/) (2026)
- [Enhancing Emotion Recognition in Incomplete Data: A Novel Cross-Modal Alignment, Reconstruction, and Refinement Framework](https://mlsanity.com/paper/2407.09029/) (2024)
- [Temporal-Spatial Decouple before Act: Disentangled Representation Learning for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2601.13659/) (2026)
- [Group Cognition Learning: Making Everything Better Through Governed Two-Stage Agents Collaboration](https://mlsanity.com/paper/2605.00370/) (2026)
- [Developing vocal system impaired patient-aimed voice quality assessment approach using ASR representation-included multiple features](https://mlsanity.com/paper/2408.12279/) (2024)
- [GIA-MIC: Multimodal Emotion Recognition with Gated Interactive Attention and Modality-Invariant Learning Constraints](https://mlsanity.com/paper/2506.00865/) (2025)

## Abstract (arXiv)

Multimodal emotion recognition (MMER) is an active research field that aims to accurately recognize human emotions by fusing multiple perceptual modalities. However, inherent heterogeneity across modalities introduces distribution gaps and information redundancy, posing significant challenges for MMER. In this paper, we propose a novel fine-grained disentangled representation learning (FDRL) framework to address these challenges. Specifically, we design modality-shared and modality-private encoders to project each modality into modality-shared and modality-private subspaces, respectively. In the shared subspace, we introduce a fine-grained alignment component to learn modality-shared representations, thus capturing modal consistency. Subsequently, we tailor a fine-grained disparity component to constrain the private subspaces, thereby learning modality-private representations and enhancing their diversity. Lastly, we introduce a fine-grained predictor component to ensure that the labels of the output representations from the encoders remain unchanged. Experimental results on the IEMOCAP dataset show that FDRL outperforms the state-of-the-art methods, achieving 78.34% and 79.44% on WAR and UAR, respectively.