Enhancing Speech Emotion Recognition with Graph-Based Multimodal Fusion and Prosodic Features for the Speech Emotion Recognition in Naturalistic Conditions Challenge at Interspeech 2025
Not on the map. Why?
Abstract (arXiv)
Training SER models in natural, spontaneous speech is especially challenging due to the subtle expression of emotions and the unpredictable nature of real-world audio. In this paper, we present a robust system for the INTERSPEECH 2025 Speech Emotion Recognition in Naturalistic Conditions Challenge, focusing on categorical emotion recognition. Our method combines state-of-the-art audio models with text features enriched by prosodic and spectral cues. In particular, we investigate the effectiveness of Fundamental Frequency (F0) quantization and the use of a pretrained audio tagging model. We also employ an ensemble model to improve robustness. On the official test set, our system achieved a Macro F1-score of 39.79% (42.20% on validation). Our results underscore the potential of these methods, and analysis of fusion techniques confirmed the effectiveness of Graph Attention Networks. Our source code is publicly available.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 16 of its 35 references that are in the map. First posted on arXiv 2 Jun 2025.
Builds on
16 of 35 references.
References · 16
- 2017Graph Attention Networkscited by 8,923
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
- 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
- 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
- 2020GLU Variants Improve Transformercited by 1,804
- 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
- 2019PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognitioncited by 574
- 2018ESPnet: End-to-End Speech Processing Toolkitcited by 558
- 2024Multilingual E5 Text Embeddings: A Technical Reportcited by 372
- 2022Speech Emotion Recognition using Self-Supervised Featurescited by 53
- 2024Towards Robust Speech Representation Learning for Thousands of Languagescited by 50
- 2023RMVPE: A Robust Model for Vocal Pitch Estimation in Polyphonic Musiccited by 30
4 more
- 2023CED: Consistent ensemble distillation for audio taggingcited by 29
- 2023HCAM -- Hierarchical Cross Attention Model for Multi-modal Emotion Recognitioncited by 11
- 2023SeqAug: Sequential Feature Resampling as a modality agnostic augmentation methodcited by 3
- 20241st Place Solution to Odyssey Emotion Recognition Challenge Task1: Tackling Class Imbalance Problemcited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Emotion as a Distribution: Joint Valence-Arousal Probability Learning for Speaker-Independent Multimodal Emotion Recognition
- 2025MERaLiON-SER: Robust Speech Emotion Recognition Model for English and SEA Languages
- 2025MSF-SER: Enriching Acoustic Modeling with Multi-Granularity Semantics for Speech Emotion Recognition ↗
- 2022Speech Emotion: Investigating Model Representations, Multi-Task Learning and Knowledge Distillation
- 2026Acoustic and Semantic Modeling of Emotion in Spoken Language ↗
- 2023Speech Emotion Recognition with Distilled Prosodic and Linguistic Affect Representations
- 2026Decoding Ambiguous Emotions with Test-Time Scaling in Audio-Language Models
- 2021Representation learning through cross-modal conditional teacher-student training for speech emotion recognition