# HeLo: Heterogeneous Multi-Modal Fusion with Label Correlation for Emotion Distribution Learning

arXiv: https://arxiv.org/abs/2507.06821 · HTML: https://mlsanity.com/paper/2507.06821/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 53 references that are in the map.
- First posted on arXiv 9 Jul 2025.

## Builds on (most cited first, 5 of 5)

- [Label Distribution Learning](https://mlsanity.com/paper/1408.6027/) (2014)
- [Deep Label Distribution Learning with Label Ambiguity](https://mlsanity.com/paper/1611.01731/) (2016)
- [Exploiting Multi-Label Correlation in Label Distribution Learning](https://mlsanity.com/paper/2308.01742/) (2023)
- [CARAT: Contrastive Feature Reconstruction and Aggregation for Multi-Modal Multi-Label Emotion Recognition](https://mlsanity.com/paper/2312.10201/) (2023)
- [Causal-aware Large Language Models: Enhancing Decision-Making Through Learning, Adapting and Acting](https://mlsanity.com/paper/2505.24710/) (2025)

## Built on by (most cited first, 1 of 1)

- [Memory-guided Prototypical Co-occurrence Learning for Mixed Emotion Recognition](https://mlsanity.com/paper/2602.20530/) (2026)

## Abstract (arXiv)

Multi-modal emotion recognition has garnered increasing attention as it plays a significant role in human-computer interaction (HCI) in recent years. Since different discrete emotions may exist at the same time, compared with single-class emotion recognition, emotion distribution learning (EDL) that identifies a mixture of basic emotions has gradually emerged as a trend. However, existing EDL methods face challenges in mining the heterogeneity among multiple modalities. Besides, rich semantic correlations across arbitrary basic emotions are not fully exploited. In this paper, we propose a multi-modal emotion distribution learning framework, named HeLo, aimed at fully exploring the heterogeneity and complementary information in multi-modal emotional data and label correlation within mixed basic emotions. Specifically, we first adopt cross-attention to effectively fuse the physiological data. Then, an optimal transport (OT)-based heterogeneity mining module is devised to mine the interaction and heterogeneity between the physiological and behavioral representations. To facilitate label correlation learning, we introduce a learnable label embedding optimized by correlation matrix alignment. Finally, the learnable label embeddings and label correlation matrices are integrated with the multi-modal representations through a novel label correlation-driven cross-attention mechanism for accurate emotion distribution learning. Experimental results on two publicly available datasets demonstrate the superiority of our proposed method in emotion distribution learning.