# Semi-Supervised Speech Emotion Recognition with Ladder Networks

arXiv: https://arxiv.org/abs/1905.02921 · HTML: https://mlsanity.com/paper/1905.02921/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 May 2019.

## Built on by (most cited first, 9 of 9)

- [Speech Emotion Recognition with Multiscale Area Attention and Data Augmentation](https://mlsanity.com/paper/2102.01813/) (2021)
- [Improving Speech Emotion Recognition in Under-Resourced Languages via Speech-to-Speech Translation with Bootstrapping Data Selection](https://mlsanity.com/paper/2409.10985/) (2024)
- [Is It Still Fair? Investigating Gender Fairness in Cross-Corpus Speech Emotion Recognition](https://mlsanity.com/paper/2501.00995/) (2025)
- [Mouth Articulation-Based Anchoring for Improved Cross-Corpus Speech Emotion Recognition](https://mlsanity.com/paper/2412.19909/) (2024)
- [Addressing Data Scarcity in Multimodal User State Recognition by Combining Semi-Supervised and Supervised Learning](https://mlsanity.com/paper/2202.03775/) (2022)
- [Integrating Contrastive Learning into a Multitask Transformer Model for Effective Domain Adaptation](https://mlsanity.com/paper/2310.04703/) (2023)
- [Joint Learning using Mixture-of-Expert-Based Representation for Speech Enhancement and Robust Emotion Recognition](https://mlsanity.com/paper/2509.08470/) (2025)
- [Speaker Style-Aware Phoneme Anchoring for Improved Cross-Lingual Speech Emotion Recognition](https://mlsanity.com/paper/2509.20373/) (2025)
- [Improving Speech Emotion Recognition with Mutual Information Regularized Generative Model](https://mlsanity.com/paper/2510.10078/) (2025)

## Abstract (arXiv)

Speech emotion recognition (SER) systems find applications in various fields such as healthcare, education, and security and defense. A major drawback of these systems is their lack of generalization across different conditions. This problem can be solved by training models on large amounts of labeled data from the target domain, which is expensive and time-consuming. Another approach is to increase the generalization of the models. An effective way to achieve this goal is by regularizing the models through multitask learning (MTL), where auxiliary tasks are learned along with the primary task. These methods often require the use of labeled data which is computationally expensive to collect for emotion recognition (gender, speaker identity, age or other emotional descriptors). This study proposes the use of ladder networks for emotion recognition, which utilizes an unsupervised auxiliary task. The primary task is a regression problem to predict emotional attributes. The auxiliary task is the reconstruction of intermediate feature representations using a denoising autoencoder. This auxiliary task does not require labels so it is possible to train the framework in a semi-supervised fashion with abundant unlabeled data from the target domain. This study shows that the proposed approach creates a powerful framework for SER, achieving superior performance than fully supervised single-task learning (STL) and MTL baselines. The approach is implemented with several acoustic features, showing that ladder networks generalize significantly better in cross-corpus settings. Compared to the STL baselines, the proposed approach achieves relative gains in concordance correlation coefficient (CCC) between 3.0% and 3.5% for within corpus evaluations, and between 16.1% and 74.1% for cross corpus evaluations, highlighting the power of the architecture.