mlsanityData to 14 Oct 2026

MERaLiON-SER: Robust Speech Emotion Recognition Model for English and SEA Languages

arXiv 2511.04914 ↗First posted 7 Nov 2025cs.SD cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

We present MERaLiON-SER, a robust speech emotion recognition model designed for English and Southeast Asian languages. The model is trained using a hybrid objective combining weighted categorical cross-entropy and Concordance Correlation Coefficient (CCC) losses for joint discrete and dimensional emotion modelling. This dual approach enables the model to capture both the distinct categories of emotion (like happy or angry) and the fine-grained, such as arousal (intensity), valence (positivity/negativity), and dominance (sense of control), leading to a more comprehensive and robust representation of human affect. Extensive evaluations across multilingual Singaporean languages (English, Chinese, Malay, and Tamil ) and other public benchmarks show that MERaLiON-SER consistently surpasses both open-source speech encoders and large Audio-LLMs. These results underscore the importance of specialised speech-only models for accurate paralinguistic understanding and cross-lingual generalisation. Furthermore, the proposed framework provides a foundation for integrating emotion-aware perception into future agentic audio systems, enabling more empathetic and contextually adaptive multimodal reasoning.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 21 of its 32 references that are in the map. First posted on arXiv 7 Nov 2025.

Builds on

21 of 32 references.

References · 21

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  3. 2018MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversationscited by 643
  4. 2020ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verificationcited by 628
  5. 2015MUSAN: A Music, Speech, and Noise Corpuscited by 425
  6. 2022Self-Supervised Speech Representation Learning: A Reviewcited by 260
  7. 2023emotion2vec: Self-Supervised Pre-Training for Speech Emotion Representationcited by 205
  8. 2024Advancing Large Language Models to Capture Varied Speaking Styles and Respond Properly in Spoken Conversationscited by 45
  9. 2024DeSTA2: Developing Instruction-Following Speech Language Model Without Speech Instruction-Tuning Datacited by 45
  10. 2024EmoBox: Multilingual Multi-corpus Speech Emotion Recognition Toolkit and Benchmarkcited by 40
  11. 2024BLSP-Emo: Towards Empathetic Large Speech-Language Modelscited by 36
  12. 2023SECap: Speech Emotion Captioning with Large Language Modelcited by 33
9 more
  1. 2024DeSTA: Enhancing Speech Language Models through Descriptive Speech-Text Alignmentcited by 33
  2. 2024Beyond Silent Letters: Amplifying LLMs in Emotion Recognition with Vocal Nuancescited by 27
  3. 2025Do Audio LLMs Really LISTEN, or Just Transcribe? Measuring Lexical vs. Acoustic Emotion Cues Reliancecited by 24
  4. 2024Frozen Large Language Models Can Perceive Paralinguistic Aspects of Speechcited by 18
  5. 2024Paralinguistics-Aware Speech-Empowered Large Language Models for Natural Conversationcited by 14
  6. 2025Benchmarking Contextual and Paralinguistic Reasoning in Speech-LLMs: A Case Study with In-the-Wild Datacited by 10
  7. 2025SeaLLMs-Audio: Large Audio-Language Models for Southeast Asiacited by 8
  8. 2024SER Evals: In-domain and Out-of-domain Benchmarking for Speech Emotion Recognitioncited by 6
  9. 2025Contextual Paralinguistic Data Creation for Multi-Modal Speech-LLM: Data Condensation and Spoken QA Generationcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Multi-Teacher Language-Aware Knowledge Distillation for Multilingual Speech Emotion Recognition
  2. 2026Decoding Ambiguous Emotions with Test-Time Scaling in Audio-Language Models
  3. 2025Pioneering Multimodal Emotion Recognition in the Era of Large Models: From Closed Sets to Open Vocabularies
  4. 2024Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction Tuning
  5. 2023Speech Emotion Recognition with Distilled Prosodic and Linguistic Affect Representations
  6. 2026Emotion as a Distribution: Joint Valence-Arousal Probability Learning for Speaker-Independent Multimodal Emotion Recognition
  7. 2025MSF-SER: Enriching Acoustic Modeling with Multi-Granularity Semantics for Speech Emotion Recognition ↗
  8. 2024MERaLiON-AudioLLM: Bridging Audio and Language with Large Language Models