Speech Emotion Recognition with Phonation Excitation Information and Articulatory Kinematics
Not on the map. Why?
Abstract (arXiv)
Speech emotion recognition (SER) has advanced significantly for the sake of deep-learning methods, while textual information further enhances its performance. However, few studies have focused on the physiological information during speech production, which also encompasses speaker traits, including emotional states. To bridge this gap, we conducted a series of experiments to investigate the potential of the phonation excitation information and articulatory kinematics for SER. Due to the scarcity of training data for this purpose, we introduce a portrayed emotional dataset, STEM-E2VA, which includes audio and physiological data such as electroglottography (EGG) and electromagnetic articulography (EMA). EGG and EMA provide information of phonation excitation and articulatory kinematics, respectively. Additionally, we performed emotion recognition using estimated physiological data derived through inversion methods from speech, instead of collected EGG and EMA, to explore the feasibility of applying such physiological information in real-world SER. Experimental results confirm the effectiveness of incorporating physiological information about speech production for SER and demonstrate its potential for practical use in real-world scenarios.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 12 of its 38 references that are in the map. First posted on arXiv 11 Nov 2025.
Builds on
12 of 38 references.
References · 12
- 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
- 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
- 2022Evidence of Vocal Tract Articulation in Self-Supervised Learning of Speechcited by 31
- 2023Self-Supervised Models of Speech Infer Universal Articulatory Kinematicscited by 16
- 2023Speaker-Independent Acoustic-to-Articulatory Speech Inversioncited by 15
- 2023Leveraging Speech PTM, Text LLM, and Emotional TTS for Speech Emotion Recognitioncited by 12
- 2024MF-AED-AEC: Speech Emotion Recognition by Leveraging Multimodal Fusion, Asr Error Detection, and Asr Error Correctioncited by 8
- 2023Improving Speech Inversion Through Self-Supervised Embeddings and Enhanced Tract Variablescited by 7
- 2022The Secret Source : Incorporating Source Features to Improve Acoustic-to-Articulatory Speech Inversioncited by 5
- 2024Speech Swin-Transformer: Exploring a Hierarchical Transformer with Shifted Windows for Speech Emotion Recognitioncited by 4
- 2018Contribution of Glottal Waveform in Speech Emotion: A Comparative Pairwise Investigationcited by 1
- 2019Adversarial Approximate Inference for Speech to Electroglottograph Conversioncited by 1
Built on by
Not cited yet.