# AmbER2: Dual Ambiguity-Aware Emotion Recognition Applied to Speech and Text

arXiv: https://arxiv.org/abs/2601.18010 · HTML: https://mlsanity.com/paper/2601.18010/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Jan 2026.

## Built on by (most cited first, 4 of 4)

- [SHALA-LLM: Smartly Handling Ambiguous Labels in Aligning LLMs](https://mlsanity.com/paper/2606.05376/) (2026)
- [Learning Emotion from Motion: Kinetic Multi-Stream Skeleton Modeling with Metadata-Conditioned Weak Label Distributions](https://mlsanity.com/paper/2607.17121/) (2026)
- [BiCFlow-MER: Orchestrating Discriminative and Generative Multimodal Emotion Recognition via Conditional Transport](https://mlsanity.com/paper/2609.27615/) (2026)
- [Whose Ground Truth? Embracing Ambiguity in Human-Centered AI](https://mlsanity.com/paper/2610.10805/) (2026)

## Abstract (arXiv)

Emotion recognition is inherently ambiguous, with uncertainty arising both from rater disagreement and from discrepancies across modalities such as speech and text. There is growing interest in modeling rater ambiguity using label distributions. However, modality ambiguity remains underexplored, and multimodal approaches often rely on simple feature fusion without explicitly addressing conflicts between modalities. In this work, we propose AmbER2, a dual ambiguity-aware framework that simultaneously models rater-level and modality-level ambiguity through a teacher-student architecture with a distribution-wise training objective. Evaluations on IEMOCAP and MSP-Podcast show that AmbER2 consistently improves distributional fidelity over conventional cross-entropy baselines and achieves performance competitive with, or superior to, recent state-of-the-art systems. For example, on IEMOCAP, AmbER2 achieves relative improvements of 20.3% on Bhattacharyya coefficient (0.83 vs. 0.69), 13.6% on R2 (0.67 vs. 0.59), 3.8% on accuracy (0.683 vs. 0.658), and 4.5% on F1 (0.675 vs. 0.646). Further analysis across ambiguity levels shows that explicitly modeling ambiguity is particularly beneficial for highly uncertain samples. These findings highlight the importance of jointly addressing rater and modality ambiguity when building robust emotion recognition systems.