# End-to-End Speech Emotion Recognition: Challenges of Real-Life Emergency Call Centers Data Recordings

arXiv: https://arxiv.org/abs/2110.14957 · HTML: https://mlsanity.com/paper/2110.14957/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 24 references that are in the map.
- First posted on arXiv 28 Oct 2021.

## Builds on (most cited first, 5 of 5)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [A Structured Self-attentive Sentence Embedding](https://mlsanity.com/paper/1703.03130/) (2017)
- [SEWA DB: A Rich Database for Audio-Visual Emotion and Sentiment Research in the Wild](https://mlsanity.com/paper/1901.02839/) (2019)
- [Multi-modal Attention for Speech Emotion Recognition](https://mlsanity.com/paper/2009.04107/) (2020)
- [Deep Architecture Enhancing Robustness to Noise, Adversarial Attacks, and Cross-corpus Setting for Speech Emotion Recognition](https://mlsanity.com/paper/2005.08453/) (2020)

## Built on by (most cited first, 8 of 8)

- [A Comprehensive Study on the Effectiveness of ASR Representations for Noise-Robust Speech Emotion Recognition](https://mlsanity.com/paper/2311.07093/) (2023)
- [Exploring Attention Mechanisms for Multimodal Emotion Recognition in an Emergency Call Center Corpus](https://mlsanity.com/paper/2306.07115/) (2023)
- [A Multi-Task, Multi-Modal Approach for Predicting Categorical and Dimensional Emotions](https://mlsanity.com/paper/2401.00536/) (2023)
- [Dynamic Stress Detection: A Study of Temporal Progression Modelling of Stress in Speech](https://mlsanity.com/paper/2510.08586/) (2025)
- [Multiscale Contextual Learning for Speech Emotion Recognition in Emergency Call Center Conversations](https://mlsanity.com/paper/2308.14894/) (2023)
- [Emotional Voice Messages (EMOVOME) database: emotion recognition in spontaneous voice messages](https://mlsanity.com/paper/2402.17496/) (2024)
- [Joint Learning using Mixture-of-Expert-Based Representation for Speech Enhancement and Robust Emotion Recognition](https://mlsanity.com/paper/2509.08470/) (2025)
- [TRIDENT: A Redundant Architecture for Caribbean-Accented Emergency Speech Triage](https://mlsanity.com/paper/2512.10741/) (2025)

## Abstract (arXiv)

Recognizing a speaker's emotion from their speech can be a key element in emergency call centers. End-to-end deep learning systems for speech emotion recognition now achieve equivalent or even better results than conventional machine learning approaches. In this paper, in order to validate the performance of our neural network architecture for emotion recognition from speech, we first trained and tested it on the widely used corpus accessible by the community, IEMOCAP. We then used the same architecture as the real life corpus, CEMO, composed of 440 dialogs (2h16m) from 485 speakers. The most frequent emotions expressed by callers in these real life emergency dialogues are fear, anger and positive emotions such as relief. In the IEMOCAP general topic conversations, the most frequent emotions are sadness, anger and happiness. Using the same end-to-end deep learning architecture, an Unweighted Accuracy Recall (UA) of 63% is obtained on IEMOCAP and a UA of 45.6% on CEMO, each with 4 classes. Using only 2 classes (Anger, Neutral), the results for CEMO are 76.9% UA compared to 81.1% UA for IEMOCAP. We expect that these encouraging results with CEMO can be improved by combining the audio channel with the linguistic channel. Real-life emotions are clearly more complex than acted ones, mainly due to the large diversity of emotional expressions of speakers. Index Terms-emotion detection, end-to-end deep learning architecture, call center, real-life database, complex emotions.