# EM2LDL: A Multilingual Speech Corpus for Mixed Emotion Recognition through Label Distribution Learning

arXiv: https://arxiv.org/abs/2511.20106 · HTML: https://mlsanity.com/paper/2511.20106/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 86 references that are in the map.
- First posted on arXiv 25 Nov 2025.

## Builds on (most cited first, 15 of 15)

- [Self-Supervised Speech Representation Learning: A Review](https://mlsanity.com/paper/2205.10643/) (2022)
- [Label Distribution Learning](https://mlsanity.com/paper/1408.6027/) (2014)
- [A Fine-tuned Wav2vec 2.0/HuBERT Benchmark For Speech Emotion Recognition, Speaker Verification and Spoken Language Understanding](https://mlsanity.com/paper/2111.02735/) (2021)
- [A Survey of Code-switched Speech and Language Processing](https://mlsanity.com/paper/1904.00784/) (2019)
- [SEWA DB: A Rich Database for Audio-Visual Emotion and Sentiment Research in the Wild](https://mlsanity.com/paper/1901.02839/) (2019)
- [Uncertainty-aware Label Distribution Learning for Facial Expression Recognition](https://mlsanity.com/paper/2209.10448/) (2022)
- [Speech Synthesis with Mixed Emotions](https://mlsanity.com/paper/2208.05890/) (2022)
- [Enhancing Code-switching Speech Recognition with Interactive Language Biases](https://mlsanity.com/paper/2309.16953/) (2023)
- [EmoNet: A Transfer Learning Framework for Multi-Corpus Speech Emotion Recognition](https://mlsanity.com/paper/2103.08310/) (2021)
- [Self Supervised Adversarial Domain Adaptation for Cross-Corpus and Cross-Language Speech Emotion Recognition](https://mlsanity.com/paper/2204.08625/) (2022)
- [AffectNet+: A Database for Enhancing Facial Expression Recognition with Soft-Labels](https://mlsanity.com/paper/2410.22506/) (2024)
- [The ACM Multimedia 2023 Computational Paralinguistics Challenge: Emotion Share & Requests](https://mlsanity.com/paper/2304.14882/) (2023)
- [End-to-End Label Uncertainty Modeling in Speech Emotion Recognition using Bayesian Neural Networks and Label Distribution Learning](https://mlsanity.com/paper/2209.15449/) (2022)
- [TrustSER: On the Trustworthiness of Fine-tuning Pre-trained Speech Embeddings For Speech Emotion Recognition](https://mlsanity.com/paper/2305.11229/) (2023)
- [Computational Sarcasm Analysis on Social Media: A Systematic Review](https://mlsanity.com/paper/2209.06170/) (2022)

## Abstract (arXiv)

This study introduces EM2LDL, a novel multilingual speech corpus designed to advance mixed emotion recognition through label distribution learning. Addressing the limitations of predominantly monolingual and single-label emotion corpora textcolorblackthat restrict linguistic diversity, are unable to model mixed emotions, and lack ecological validity, EM2LDL comprises expressive utterances in English, Mandarin, and Cantonese, capturing the intra-utterance code-switching prevalent in multilingual regions like Hong Kong and Macao. The corpus integrates spontaneous emotional expressions from online platforms, annotated with fine-grained emotion distributions across 32 categories. Experimental baselines using self-supervised learning models demonstrate robust performance in speaker-independent gender-, age-, and personality-based evaluations, with HuBERT-large-EN achieving optimal results. By incorporating linguistic diversity and ecological validity, EM2LDL enables the exploration of complex emotional dynamics in multilingual settings. This work provides a versatile testbed for developing adaptive, empathetic systems for applications in affective computing, including mental health monitoring and cross-cultural communication. The dataset, annotations, and baseline codes are publicly available at https://github.com/xingfengli/EM2LDL.