# Towards Speech Emotion Recognition "in the wild" using Aggregated Corpora and Deep Multi-Task Learning

arXiv: https://arxiv.org/abs/1708.03920 · HTML: https://mlsanity.com/paper/1708.03920/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Aug 2017.

## Built on by (most cited first, 6 of 6)

- [Pareto Set Learning for Multi-Objective Reinforcement Learning](https://mlsanity.com/paper/2501.06773/) (2025)
- [EmoNet: A Transfer Learning Framework for Multi-Corpus Speech Emotion Recognition](https://mlsanity.com/paper/2103.08310/) (2021)
- [A cross-corpus study on speech emotion recognition](https://mlsanity.com/paper/2207.02104/) (2022)
- [Semi-supervised cross-lingual speech emotion recognition](https://mlsanity.com/paper/2207.06767/) (2022)
- [GMP-TL: Gender-augmented Multi-scale Pseudo-label Enhanced Transfer Learning for Speech Emotion Recognition](https://mlsanity.com/paper/2405.02151/) (2024)
- [On the Efficacy and Noise-Robustness of Jointly Learned Speech Emotion and Automatic Speech Recognition](https://mlsanity.com/paper/2305.12540/) (2023)

## Abstract (arXiv)

One of the challenges in Speech Emotion Recognition (SER) "in the wild" is the large mismatch between training and test data (e.g. speakers and tasks). In order to improve the generalisation capabilities of the emotion models, we propose to use Multi-Task Learning (MTL) and use gender and naturalness as auxiliary tasks in deep neural networks. This method was evaluated in within-corpus and various cross-corpus classification experiments that simulate conditions "in the wild". In comparison to Single-Task Learning (STL) based state of the art methods, we found that our MTL method proposed improved performance significantly. Particularly, models using both gender and naturalness achieved more gains than those using either gender or naturalness separately. This benefit was also found in the high-level representations of the feature space, obtained from our method proposed, where discriminative emotional clusters could be observed.