# Speaker Attentive Speech Emotion Recognition

arXiv: https://arxiv.org/abs/2104.07288 · HTML: https://mlsanity.com/paper/2104.07288/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 23 references that are in the map.
- First posted on arXiv 15 Apr 2021.

## Builds on (most cited first, 6 of 6)

- [VoxCeleb: a large-scale speaker identification dataset](https://mlsanity.com/paper/1706.08612/) (2017)
- [Seen and Unseen emotional style transfer for voice conversion with a new emotional speech dataset](https://mlsanity.com/paper/2010.14794/) (2020)
- [Multi-modal Attention for Speech Emotion Recognition](https://mlsanity.com/paper/2009.04107/) (2020)
- [Attentive Convolutional Neural Network based Speech Emotion Recognition: A Study on the Impact of Input Features, Signal Length, and Acted Speech](https://mlsanity.com/paper/1706.00612/) (2017)
- [Convolutional RNN: an Enhanced Model for Extracting Features from Sequential Data](https://mlsanity.com/paper/1602.05875/) (2016)
- [Att-HACK: An Expressive Speech Database with Social Attitudes](https://mlsanity.com/paper/2004.04410/) (2020)

## Built on by (most cited first, 9 of 9)

- [Speaker Normalization for Self-supervised Speech Emotion Recognition](https://mlsanity.com/paper/2202.01252/) (2022)
- [Designing and Evaluating Speech Emotion Recognition Systems: A reality check case study with IEMOCAP](https://mlsanity.com/paper/2304.00860/) (2023)
- [Exploring speaker enrolment for few-shot personalisation in emotional vocalisation prediction](https://mlsanity.com/paper/2206.06680/) (2022)
- [A Comparative Study of Pre-trained Speech and Audio Embeddings for Speech Emotion Recognition](https://mlsanity.com/paper/2304.11472/) (2023)
- [Enrolment-based personalisation for improving individual-level fairness in speech emotion recognition](https://mlsanity.com/paper/2406.06665/) (2024)
- [SPEAKER VGG CCT: Cross-corpus Speech Emotion Recognition with Speaker Embedding and Vision Transformers](https://mlsanity.com/paper/2211.02366/) (2022)
- [Exploring Multilingual Unseen Speaker Emotion Recognition: Leveraging Co-Attention Cues in Multitask Learning](https://mlsanity.com/paper/2406.08931/) (2024)
- [Meta-PerSER: Few-Shot Listener Personalized Speech Emotion Recognition via Meta-learning](https://mlsanity.com/paper/2505.16220/) (2025)
- [Towards a Common Speech Analysis Engine](https://mlsanity.com/paper/2203.00613/) (2022)

## Abstract (arXiv)

Speech Emotion Recognition (SER) task has known significant improvements over the last years with the advent of Deep Neural Networks (DNNs). However, even the most successful methods are still rather failing when adaptation to specific speakers and scenarios is needed, inevitably leading to poorer performances when compared to humans. In this paper, we present novel work based on the idea of teaching the emotion recognition network about speaker identity. Our system is a combination of two ACRNN classifiers respectively dedicated to speaker and emotion recognition. The first informs the latter through a Self Speaker Attention (SSA) mechanism that is shown to considerably help to focus on emotional information of the speech signal. Experiments on social attitudes database Att-HACK and IEMOCAP corpus demonstrate the effectiveness of the proposed method and achieve the state-of-the-art performance in terms of unweighted average recall.