VoxEmo: Benchmarking Speech Emotion Recognition with Speech LLMs
Not on the map. Why?
Abstract (arXiv)
Speech Large Language Models (LLMs) show great promise for speech emotion recognition (SER) via generative interfaces. However, shifting from closed-set classification to open text generation introduces zero-shot stochasticity, making evaluation highly sensitive to prompts. Additionally, conventional speech LLMs benchmarks overlook the inherent ambiguity of human emotion. Hence, we present VoxEmo, a comprehensive SER benchmark encompassing 35 emotion corpora across 15 languages for Speech LLMs. VoxEmo provides a standardized toolkit featuring varying prompt complexities, from direct classification to paralinguistic reasoning. To reflect real-world perception/application, we introduce a distribution-aware soft-label protocol and a prompt-ensemble strategy that emulates annotator disagreement. Experiments reveal that while zero-shot speech LLMs trail supervised baselines in hard-label accuracy, they uniquely align with human subjective distributions.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 28 of its 62 references that are in the map. First posted on arXiv 9 Mar 2026.
Builds on
28 of 62 references.
References · 28
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
- 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 652
- 2018MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversationscited by 636
- 2024Qwen2-Audio Technical Reportcited by 543
- 2025Audio Flamingo 3: Advancing Audio Intelligence with Fully Open Large Audio Language Modelscited by 210
- 2021Emotion Recognition from Speech Using Wav2vec 2.0 Embeddingscited by 158
- 2022Dawn of the transformer era in speech emotion recognition: closing the valence gapcited by 155
- 2020Seen and Unseen emotional style transfer for voice conversion with a new emotional speech datasetcited by 119
- 2024Recent Advances in Speech Language Models: A Surveycited by 85
- 2018The Emotional Voices Database: Towards Controlling the Emotion Dimension in Voice Generation Systemscited by 61
- 2022M3ED: Multi-modal Multi-scene Multi-label Emotional Dialogue Databasecited by 50
16 more
- 2024EmoBox: Multilingual Multi-corpus Speech Emotion Recognition Toolkit and Benchmarkcited by 39
- 2018Cross Lingual Speech Emotion Recognition: Urdu vs. Western Languagescited by 25
- 2019ShEMO -- A Large-Scale Validated Database for Persian Speech Emotion Detectioncited by 21
- 2025The MSP-Podcast Corpuscited by 14
- 2022A cross-corpus study on speech emotion recognitioncited by 7
- 2023EMNS /Imz/ Corpus: An emotive single-speaker dataset for narrative storytelling in games, television and graphic novelscited by 7
- 2024On Prompt Sensitivity of ChatGPT in Affective Computingcited by 7
- 2025OmniVox: Zero-Shot Emotion Recognition with Omni-LLMscited by 7
- 2025EmotionTalk: An Interactive Chinese Multimodal Emotion Dataset With Rich Annotationscited by 7
- 2022A New Amharic Speech Emotion Dataset and Classification Benchmarkcited by 5
- 2024nEMO: Dataset of Emotional Speech in Polishcited by 3
- 2025THAI Speech Emotion Recognition (THAI-SER) corpuscited by 3
- 2025Reasoning Beyond Majority Vote: An Explainable SpeechLM Framework for Speech Emotion Recognitioncited by 2
- 2025Revisiting Modeling and Evaluation Approaches in Speech Emotion Recognition: Considering Subjectivity of Annotators and Ambiguity of Emotionscited by 2
- 2026Scaling Ambiguity: Augmenting Human Annotation in Speech Emotion Recognition with Audio-Language Modelscited by 2
- 2025Rethinking Cross-Corpus Speech Emotion Recognition Benchmarking: Are Paralinguistic Pre-Trained Representations Sufficient?cited by 1
Built on by
Citing papers per year: 2026: 2.
2 papers.