# Real-time Speech Emotion Recognition Based on Syllable-Level Feature Extraction

arXiv: https://arxiv.org/abs/2204.11382 · HTML: https://mlsanity.com/paper/2204.11382/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 51 references that are in the map.
- First posted on arXiv 25 Apr 2022.

## Builds on (most cited first, 11 of 11)

- [TensorFlow.js: Machine Learning for the Web and Beyond](https://mlsanity.com/paper/1901.05350/) (2019)
- [Domain Adversarial for Acoustic Emotion Recognition](https://mlsanity.com/paper/1804.07690/) (2018)
- [Multi-Modal Emotion recognition on IEMOCAP Dataset using Deep Learning](https://mlsanity.com/paper/1804.05788/) (2018)
- [On the use of Self-supervised Pre-trained Acoustic and Linguistic Features for Continuous Speech Emotion Recognition](https://mlsanity.com/paper/2011.09212/) (2020)
- [Attentive Convolutional Neural Network based Speech Emotion Recognition: A Study on the Impact of Input Features, Signal Length, and Acted Speech](https://mlsanity.com/paper/1706.00612/) (2017)
- [Augmenting Generative Adversarial Networks for Speech Emotion Recognition](https://mlsanity.com/paper/2005.08447/) (2020)
- [Deep Architecture Enhancing Robustness to Noise, Adversarial Attacks, and Cross-corpus Setting for Speech Emotion Recognition](https://mlsanity.com/paper/2005.08453/) (2020)
- [Emotion Recognition From Speech With Recurrent Neural Networks](https://mlsanity.com/paper/1701.08071/) (2017)
- [Cross Lingual Cross Corpus Speech Emotion Recognition](https://mlsanity.com/paper/2003.07996/) (2020)
- [Reusing Neural Speech Representations for Auditory Emotion Recognition](https://mlsanity.com/paper/1803.11508/) (2018)
- [ConcealNet: An End-to-end Neural Network for Packet Loss Concealment in Deep Speech Emotion Recognition](https://mlsanity.com/paper/2005.07777/) (2020)

## Built on by (most cited first, 2 of 2)

- [Evaluating raw waveforms with deep learning frameworks for speech emotion recognition](https://mlsanity.com/paper/2307.02820/) (2023)
- [Transformer Redesign for Late Fusion of Audio-Text Features on Ultra-Low-Power Edge Hardware](https://mlsanity.com/paper/2510.18036/) (2025)

## Abstract (arXiv)

Speech emotion recognition systems have high prediction latency because of the high computational requirements for deep learning models and low generalizability mainly because of the poor reliability of emotional measurements across multiple corpora. To solve these problems, we present a speech emotion recognition system based on a reductionist approach of decomposing and analyzing syllable-level features. Mel-spectrogram of an audio stream is decomposed into syllable-level components, which are then analyzed to extract statistical features. The proposed method uses formant attention, noise-gate filtering, and rolling normalization contexts to increase feature processing speed and tolerance to adversity. A set of syllable-level formant features is extracted and fed into a single hidden layer neural network that makes predictions for each syllable as opposed to the conventional approach of using a sophisticated deep learner to make sentence-wide predictions. The syllable level predictions help to achieve the real-time latency and lower the aggregated error in utterance level cross-corpus predictions. The experiments on IEMOCAP (IE), MSP-Improv (MI), and RAVDESS (RA) databases show that the method archives real-time latency while predicting with state-of-the-art cross-corpus unweighted accuracy of 47.6% for IE to MI and 56.2% for MI to IE.