mlsanityData to 14 Oct 2026

Steering Language Model to Stable Speech Emotion Recognition via Contextual Perception and Chain of Thought

arXiv 2502.18186 ↗First posted 25 Feb 2025cs.SD cs.CL eess.AScited by 5

On the map: LLM reasoning → Audio-language model reasoning

Abstract (arXiv)

Large-scale audio language models (ALMs), such as Qwen2-Audio, are capable of comprehending diverse audio signal, performing audio analysis and generating textual responses. However, in speech emotion recognition (SER), ALMs often suffer from hallucinations, resulting in misclassifications or irrelevant outputs. To address these challenges, we propose C2SER, a novel ALM designed to enhance the stability and accuracy of SER through Contextual perception and Chain of Thought (CoT). C2SER integrates the Whisper encoder for semantic perception and Emotion2Vec-S for acoustic perception, where Emotion2Vec-S extends Emotion2Vec with semi-supervised learning to enhance emotional discrimination. Additionally, C2SER employs a CoT approach, processing SER in a step-by-step manner while leveraging speech content and speaking styles to improve recognition. To further enhance stability, C2SER introduces self-distillation from explicit CoT to implicit CoT, mitigating error accumulation and boosting recognition accuracy. Extensive experiments show that C2SER outperforms existing popular ALMs, such as Qwen2-Audio and SECap, delivering more stable and precise emotion recognition. We release the training code, checkpoints, and test sets to facilitate further research.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Audio-language model reasoning, one of 17 directions in LLM reasoning. Builds on 38 of its 60 references that are in the map. First posted on arXiv 25 Feb 2025.

Builds on

38 of 60 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Internalizing ASR with Implicit Chain of Thought for Efficient Speech-to-Speech Conversational LLM25 Sep
  2. 2024GAMA: A Large Audio-Language Model with Advanced Audio Understanding and Complex Reasoning Abilities17 Jun

Other references · 36

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  4. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  5. 2023A Survey of Large Language Modelscited by 2,576
  6. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  7. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
  8. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,121
  9. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 735
  10. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 727
  11. 2018MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversationscited by 643
  12. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 608
24 more
  1. 2024Qwen2-Audio Technical Reportcited by 560
  2. 2023SALMONN: Towards Generic Hearing Abilities for Large Language Modelscited by 506
  3. 2023MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuningcited by 487
  4. 2024From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Stepcited by 235
  5. 2024AIR-Bench: Benchmarking Large Audio-Language Models via Generative Comprehensioncited by 220
  6. 2023emotion2vec: Self-Supervised Pre-Training for Speech Emotion Representationcited by 205
  7. 2024Emilia: An Extensive, Multilingual, and Diverse Speech Dataset for Large-Scale Speech Generationcited by 204
  8. 2024AudioBench: A Universal Benchmark for Audio Large Language Modelscited by 128
  9. 2024FunAudioLLM: Voice Understanding and Generation Foundation Models for Natural Interaction Between Humans and LLMscited by 122
  10. 2020Seen and Unseen emotional style transfer for voice conversion with a new emotional speech datasetcited by 121
  11. 2022Efficient Self-supervised Learning with Contextualized Target Representations for Vision, Speech and Languagecited by 110
  12. 2018The Emotional Voices Database: Towards Controlling the Emotion Dimension in Voice Generation Systemscited by 61
  13. 2024Auffusion: Leveraging the Power of Diffusion and Large Language Models for Text-to-Audio Generationcited by 54
  14. 2022M3ED: Multi-modal Multi-scene Multi-label Emotional Dialogue Databasecited by 50
  15. 2021Exploring Wav2vec 2.0 fine-tuning for improved speech emotion recognitioncited by 49
  16. 2024EmoBox: Multilingual Multi-corpus Speech Emotion Recognition Toolkit and Benchmarkcited by 40
  17. 2023SECap: Speech Emotion Captioning with Large Language Modelcited by 33
  18. 2024DeSTA: Enhancing Speech Language Models through Descriptive Speech-Text Alignmentcited by 33
  19. 2024Benchmarking Open-ended Audio Dialogue Understanding for Large Audio-Language Modelscited by 29
  20. 2025Audio Large Language Models Can Be Descriptive Speech Quality Evaluatorscited by 25
  21. 2024MER 2024: Semi-Supervised Learning, Noise Robustness, and Open-Vocabulary Multimodal Emotion Recognitioncited by 23
  22. 2024Frozen Large Language Models Can Perceive Paralinguistic Aspects of Speechcited by 18
  23. 2025OSUM: Advancing Open Speech Understanding Models with Limited Resources in Academiacited by 13
  24. 2024Derivative-Free Optimization for Low-Rank Adaptation in Large Language Modelscited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 4.

5 papers.

  1. 2026Speech LLMs are Contextual Reasoning Transcriberscited by 2
  2. 2025Plug-and-Play Emotion Graphs for Compositional Prompting in Zero-Shot Speech Emotion Recognitioncited by 0
  3. 2026On the Emotion Understanding of Synthesized Speechcited by 0
  4. 2026Prompt Amplification and Zero-Shot Late Fusion in Audio-Language Models for Speech Emotion Recognitioncited by 0
  5. 2026HyPASE: Hyperbolic Geometry for Parameter-Efficient Speech Emotion Fine-Tuning Framework for Large Audio-Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025MERaLiON-SER: Robust Speech Emotion Recognition Model for English and SEA Languages
  2. 2026Decoding Ambiguous Emotions with Test-Time Scaling in Audio-Language Models
  3. 2022Fast Yet Effective Speech Emotion Recognition with Self-distillation
  4. 2025Dynamic Parameter Memory: Temporary LoRA-Enhanced LLM for Long-Sequence Emotion Recognition in Conversation
  5. 2026Acoustic Cue Alignment in Audio Language Models for Speech Emotion Recognition
  6. 2026AcoustEmo: Open-Vocabulary Emotion Reasoning via Utterance-Aware Acoustic Q-Former ↗
  7. 2023Multi-Level Knowledge Distillation for Speech Emotion Recognition in Noisy Conditions
  8. 2025Behind the Scenes: Mechanistic Interpretability of LoRA-adapted Whisper for Speech Emotion Recognition