# VoxEmo: Benchmarking Speech Emotion Recognition with Speech LLMs

arXiv: https://arxiv.org/abs/2603.08936 · HTML: https://mlsanity.com/paper/2603.08936/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 62 references that are in the map.
- First posted on arXiv 9 Mar 2026.

## Builds on (most cited first, 28 of 28)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [SUPERB: Speech processing Universal PERformance Benchmark](https://mlsanity.com/paper/2105.01051/) (2021)
- [MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations](https://mlsanity.com/paper/1810.02508/) (2018)
- [Qwen2-Audio Technical Report](https://mlsanity.com/paper/2407.10759/) (2024)
- [Audio Flamingo 3: Advancing Audio Intelligence with Fully Open Large Audio Language Models](https://mlsanity.com/paper/2507.08128/) (2025)
- [Emotion Recognition from Speech Using Wav2vec 2.0 Embeddings](https://mlsanity.com/paper/2104.03502/) (2021)
- [Dawn of the transformer era in speech emotion recognition: closing the valence gap](https://mlsanity.com/paper/2203.07378/) (2022)
- [Seen and Unseen emotional style transfer for voice conversion with a new emotional speech dataset](https://mlsanity.com/paper/2010.14794/) (2020)
- [Recent Advances in Speech Language Models: A Survey](https://mlsanity.com/paper/2410.03751/) (2024)
- [The Emotional Voices Database: Towards Controlling the Emotion Dimension in Voice Generation Systems](https://mlsanity.com/paper/1806.09514/) (2018)
- [M3ED: Multi-modal Multi-scene Multi-label Emotional Dialogue Database](https://mlsanity.com/paper/2205.10237/) (2022)
- [EmoBox: Multilingual Multi-corpus Speech Emotion Recognition Toolkit and Benchmark](https://mlsanity.com/paper/2406.07162/) (2024)
- [Cross Lingual Speech Emotion Recognition: Urdu vs. Western Languages](https://mlsanity.com/paper/1812.10411/) (2018)
- [ShEMO -- A Large-Scale Validated Database for Persian Speech Emotion Detection](https://mlsanity.com/paper/1906.01155/) (2019)
- [The MSP-Podcast Corpus](https://mlsanity.com/paper/2509.09791/) (2025)
- [EmotionTalk: An Interactive Chinese Multimodal Emotion Dataset With Rich Annotations](https://mlsanity.com/paper/2505.23018/) (2025)
- [A cross-corpus study on speech emotion recognition](https://mlsanity.com/paper/2207.02104/) (2022)
- [EMNS /Imz/ Corpus: An emotive single-speaker dataset for narrative storytelling in games, television and graphic novels](https://mlsanity.com/paper/2305.13137/) (2023)
- [On Prompt Sensitivity of ChatGPT in Affective Computing](https://mlsanity.com/paper/2403.14006/) (2024)
- [OmniVox: Zero-Shot Emotion Recognition with Omni-LLMs](https://mlsanity.com/paper/2503.21480/) (2025)
- [A New Amharic Speech Emotion Dataset and Classification Benchmark](https://mlsanity.com/paper/2201.02710/) (2022)
- [nEMO: Dataset of Emotional Speech in Polish](https://mlsanity.com/paper/2404.06292/) (2024)
- [THAI Speech Emotion Recognition (THAI-SER) corpus](https://mlsanity.com/paper/2507.09618/) (2025)
- [Reasoning Beyond Majority Vote: An Explainable SpeechLM Framework for Speech Emotion Recognition](https://mlsanity.com/paper/2509.24187/) (2025)
- [Revisiting Modeling and Evaluation Approaches in Speech Emotion Recognition: Considering Subjectivity of Annotators and Ambiguity of Emotions](https://mlsanity.com/paper/2510.05934/) (2025)
- [Scaling Ambiguity: Augmenting Human Annotation in Speech Emotion Recognition with Audio-Language Models](https://mlsanity.com/paper/2601.14620/) (2026)
- [Rethinking Cross-Corpus Speech Emotion Recognition Benchmarking: Are Paralinguistic Pre-Trained Representations Sufficient?](https://mlsanity.com/paper/2509.16182/) (2025)

## Built on by (most cited first, 2 of 2)

- [Prompt Amplification and Zero-Shot Late Fusion in Audio-Language Models for Speech Emotion Recognition](https://mlsanity.com/paper/2603.23057/) (2026)
- [From Speech to Editable Concepts: Probing Emotion Recognition with Concept Bottleneck Models](https://mlsanity.com/paper/2609.39453/) (2026)

## Abstract (arXiv)

Speech Large Language Models (LLMs) show great promise for speech emotion recognition (SER) via generative interfaces. However, shifting from closed-set classification to open text generation introduces zero-shot stochasticity, making evaluation highly sensitive to prompts. Additionally, conventional speech LLMs benchmarks overlook the inherent ambiguity of human emotion. Hence, we present VoxEmo, a comprehensive SER benchmark encompassing 35 emotion corpora across 15 languages for Speech LLMs. VoxEmo provides a standardized toolkit featuring varying prompt complexities, from direct classification to paralinguistic reasoning. To reflect real-world perception/application, we introduce a distribution-aware soft-label protocol and a prompt-ensemble strategy that emulates annotator disagreement. Experiments reveal that while zero-shot speech LLMs trail supervised baselines in hard-label accuracy, they uniquely align with human subjective distributions.