# SERAB: A multi-lingual benchmark for speech emotion recognition

arXiv: https://arxiv.org/abs/2110.03414 · HTML: https://mlsanity.com/paper/2110.03414/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 36 references that are in the map.
- First posted on arXiv 7 Oct 2021.

## Builds on (most cited first, 15 of 15)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [CvT: Introducing Convolutions to Vision Transformers](https://mlsanity.com/paper/2103.15808/) (2021)
- [CNN Architectures for Large-Scale Audio Classification](https://mlsanity.com/paper/1609.09430/) (2016)
- [A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmark](https://mlsanity.com/paper/1910.04867/) (2019)
- [YouTube-8M: A Large-Scale Video Classification Benchmark](https://mlsanity.com/paper/1609.08675/) (2016)
- [BYOL for Audio: Self-Supervised Learning for General-Purpose Audio Representation](https://mlsanity.com/paper/2103.06695/) (2021)
- [Towards Learning a Universal Non-Semantic Representation of Speech](https://mlsanity.com/paper/2002.12764/) (2020)
- [The INTERSPEECH 2021 Computational Paralinguistics Challenge: COVID-19 Cough, COVID-19 Speech, Escalation & Primates](https://mlsanity.com/paper/2102.13468/) (2021)
- [LSSED: a large-scale dataset and benchmark for speech emotion recognition](https://mlsanity.com/paper/2102.01754/) (2021)
- [openXBOW - Introducing the Passau Open-Source Crossmodal Bag-of-Words Toolkit](https://mlsanity.com/paper/1605.06778/) (2016)
- [On the Bias Against Inductive Biases](https://mlsanity.com/paper/2105.14077/) (2021)

## Built on by (most cited first, 22 of 22)

- [EmoBox: Multilingual Multi-corpus Speech Emotion Recognition Toolkit and Benchmark](https://mlsanity.com/paper/2406.07162/) (2024)
- [Masked Modeling Duo: Learning Representations by Encouraging Both Networks to Model the Input](https://mlsanity.com/paper/2210.14648/) (2022)
- [BYOL-S: Learning Self-supervised Speech Representations by Bootstrapping](https://mlsanity.com/paper/2206.12038/) (2022)
- [Self-Supervised Learning for Speech Enhancement through Synthesis](https://mlsanity.com/paper/2211.02542/) (2022)
- [EnCodecMAE: Leveraging neural codecs for universal audio representation learning](https://mlsanity.com/paper/2309.07391/) (2023)
- [Self-supervised Graphs for Audio Representation Learning with Limited Labeled Data](https://mlsanity.com/paper/2202.00097/) (2022)
- [Hybrid Handcrafted and Learnable Audio Representation for Analysis of Speech Under Cognitive and Physical Load](https://mlsanity.com/paper/2203.16637/) (2022)
- [The Efficacy of Self-Supervised Speech Models for Audio Representations](https://mlsanity.com/paper/2209.12900/) (2022)
- [SER Evals: In-domain and Out-of-domain Benchmarking for Speech Emotion Recognition](https://mlsanity.com/paper/2408.07851/) (2024)
- [Decoding Emotions: A comprehensive Multilingual Study of Speech Models for Speech Emotion Recognition](https://mlsanity.com/paper/2308.08713/) (2023)
- [Semi-supervised cross-lingual speech emotion recognition](https://mlsanity.com/paper/2207.06767/) (2022)
- [EmoNet-Voice: A Fine-Grained, Expert-Verified Benchmark for Speech Emotion Detection](https://mlsanity.com/paper/2506.09827/) (2025)
- [Efficient Speech Quality Assessment using Self-supervised Framewise Embeddings](https://mlsanity.com/paper/2211.06646/) (2022)
- [A Persian ASR-based SER: Modification of Sharif Emotional Speech Database and Investigation of Persian Text Corpora](https://mlsanity.com/paper/2211.09956/) (2022)
- [Predicting Heart Activity from Speech using Data-driven and Knowledge-based features](https://mlsanity.com/paper/2406.06341/) (2024)
- [Exploring Multilingual Unseen Speaker Emotion Recognition: Leveraging Co-Attention Cues in Multitask Learning](https://mlsanity.com/paper/2406.08931/) (2024)
- [Multilingual Speech Emotion Recognition With Multi-Gating Mechanism and Neural Architecture Search](https://mlsanity.com/paper/2211.08237/) (2022)
- [deep learning of segment-level feature representation for speech emotion recognition in conversations](https://mlsanity.com/paper/2302.02419/) (2023)
- [Are Paralinguistic Representations all that is needed for Speech Emotion Recognition?](https://mlsanity.com/paper/2402.01579/) (2024)
- [Speaker Embeddings as Individuality Proxy for Voice Stress Detection](https://mlsanity.com/paper/2306.05915/) (2023)
- [Capturing Spectral and Long-term Contextual Information for Speech Emotion Recognition Using Deep Learning Techniques](https://mlsanity.com/paper/2308.04517/) (2023)
- [VoiceNet: Fine-Grained Voice Understanding Beyond Emotion at Scale](https://mlsanity.com/paper/2609.32016/) (2026)

## Abstract (arXiv)

Recent developments in speech emotion recognition (SER) often leverage deep neural networks (DNNs). Comparing and benchmarking different DNN models can often be tedious due to the use of different datasets and evaluation protocols. To facilitate the process, here, we present the Speech Emotion Recognition Adaptation Benchmark (SERAB), a framework for evaluating the performance and generalization capacity of different approaches for utterance-level SER. The benchmark is composed of nine datasets for SER in six languages. Since the datasets have different sizes and numbers of emotional classes, the proposed setup is particularly suitable for estimating the generalization capacity of pre-trained DNN-based feature extractors. We used the proposed framework to evaluate a selection of standard hand-crafted feature sets and state-of-the-art DNN representations. The results highlight that using only a subset of the data included in SERAB can result in biased evaluation, while compliance with the proposed protocol can circumvent this issue.