# Designing and Evaluating Speech Emotion Recognition Systems: A reality check case study with IEMOCAP

arXiv: https://arxiv.org/abs/2304.00860 · HTML: https://mlsanity.com/paper/2304.00860/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 32 references that are in the map.
- First posted on arXiv 3 Apr 2023.

## Builds on (most cited first, 19 of 19)

- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [SUPERB: Speech processing Universal PERformance Benchmark](https://mlsanity.com/paper/2105.01051/) (2021)
- [A Metric Learning Reality Check](https://mlsanity.com/paper/2003.08505/) (2020)
- [Emotion Recognition from Speech Using Wav2vec 2.0 Embeddings](https://mlsanity.com/paper/2104.03502/) (2021)
- [Towards Learning a Universal Non-Semantic Representation of Speech](https://mlsanity.com/paper/2002.12764/) (2020)
- [A Fine-tuned Wav2vec 2.0/HuBERT Benchmark For Speech Emotion Recognition, Speaker Verification and Spoken Language Understanding](https://mlsanity.com/paper/2111.02735/) (2021)
- [Universal Paralinguistic Speech Representations Using Self-Supervised Conformers](https://mlsanity.com/paper/2110.04621/) (2021)
- [Speech Emotion Recognition with Co-Attention based Multi-level Acoustic Information](https://mlsanity.com/paper/2203.15326/) (2022)
- [TRILLsson: Distilled Universal Paralinguistic Speech Representations](https://mlsanity.com/paper/2203.00236/) (2022)
- [Efficient Speech Emotion Recognition Using Multi-Scale CNN and Attention](https://mlsanity.com/paper/2106.04133/) (2021)
- [Light-SERNet: A lightweight fully convolutional neural network for speech emotion recognition](https://mlsanity.com/paper/2110.03435/) (2021)
- [Speaker Normalization for Self-supervised Speech Emotion Recognition](https://mlsanity.com/paper/2202.01252/) (2022)
- [Speech Emotion Recognition with Multiscale Area Attention and Data Augmentation](https://mlsanity.com/paper/2102.01813/) (2021)
- [CopyPaste: An Augmentation Method for Speech Emotion Recognition](https://mlsanity.com/paper/2010.14602/) (2020)
- [Speaker Attentive Speech Emotion Recognition](https://mlsanity.com/paper/2104.07288/) (2021)
- [Emotion Recognition From Speech With Recurrent Neural Networks](https://mlsanity.com/paper/1701.08071/) (2017)
- [Speech Emotion Recognition with Global-Aware Fusion on Multi-scale Feature Representation](https://mlsanity.com/paper/2204.05571/) (2022)

## Built on by (most cited first, 6 of 6)

- [EmoBox: Multilingual Multi-corpus Speech Emotion Recognition Toolkit and Benchmark](https://mlsanity.com/paper/2406.07162/) (2024)
- [Exploring Multilingual Unseen Speaker Emotion Recognition: Leveraging Co-Attention Cues in Multitask Learning](https://mlsanity.com/paper/2406.08931/) (2024)
- [A Multi-Task, Multi-Modal Approach for Predicting Categorical and Dimensional Emotions](https://mlsanity.com/paper/2401.00536/) (2023)
- [Variational decomposition autoencoding improves disentanglement of latent representations](https://mlsanity.com/paper/2601.06844/) (2026)
- [Beyond the Labels: Unveiling Text-Dependency in Paralinguistic Speech Recognition Datasets](https://mlsanity.com/paper/2403.07767/) (2024)
- [Acoustic Cue Alignment in Audio Language Models for Speech Emotion Recognition](https://mlsanity.com/paper/2606.07309/) (2026)

## Abstract (arXiv)

There is an imminent need for guidelines and standard test sets to allow direct and fair comparisons of speech emotion recognition (SER). While resources, such as the Interactive Emotional Dyadic Motion Capture (IEMOCAP) database, have emerged as widely-adopted reference corpora for researchers to develop and test models for SER, published work reveals a wide range of assumptions and variety in its use that challenge reproducibility and generalization. Based on a critical review of the latest advances in SER using IEMOCAP as the use case, our work aims at two contributions: First, using an analysis of the recent literature, including assumptions made and metrics used therein, we provide a set of SER evaluation guidelines. Second, using recent publications with open-sourced implementations, we focus on reproducibility assessment in SER.