# Curriculum Learning for Speech Emotion Recognition from Crowdsourced Labels

arXiv: https://arxiv.org/abs/1805.10339 · HTML: https://mlsanity.com/paper/1805.10339/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 May 2018.

## Built on by (most cited first, 16 of 16)

- [Curriculum Learning: A Survey](https://mlsanity.com/paper/2101.10382/) (2021)
- [A Comprehensive Survey on Multi-modal Conversational Emotion Recognition with Deep Learning](https://mlsanity.com/paper/2312.05735/) (2023)
- [Learning to Learn: How to Continuously Teach Humans and Machines](https://mlsanity.com/paper/2211.15470/) (2022)
- [Improving multi-speaker TTS prosody variance with a residual encoder and normalizing flows](https://mlsanity.com/paper/2106.05762/) (2021)
- [Knowledge Transfer For On-Device Speech Emotion Recognition with Neural Structured Learning](https://mlsanity.com/paper/2210.14977/) (2022)
- [Does the Definition of Difficulty Matter? Scoring Functions and their Role for Curriculum Learning](https://mlsanity.com/paper/2411.00973/) (2024)
- [Task Phasing: Automated Curriculum Learning from Demonstrations](https://mlsanity.com/paper/2210.10999/) (2022)
- [A Comprehensive Study of Privacy Risks in Curriculum Learning](https://mlsanity.com/paper/2310.10124/) (2023)
- [Meta-Learn Unimodal Signals with Weak Supervision for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2408.16029/) (2024)
- [Training speech emotion classifier without categorical annotations](https://mlsanity.com/paper/2210.07642/) (2022)
- [On the Perception of Difficulty: Differences between Humans and AI](https://mlsanity.com/paper/2304.09803/) (2023)
- [ERNetCL: A novel emotion recognition network in textual conversation based on curriculum learning strategy](https://mlsanity.com/paper/2308.06450/) (2023)
- [Adaptively Point-weighting Curriculum Learning](https://mlsanity.com/paper/2505.01665/) (2025)
- [CHUCKLE -- When Humans Teach AI To Learn Emotions The Easy Way](https://mlsanity.com/paper/2510.09382/) (2025)
- [Detecting the Severity of Major Depressive Disorder from Speech: A Novel HARD-Training Methodology](https://mlsanity.com/paper/2206.01542/) (2022)
- [ParA-LLM: A Unified Approach to Paralinguistic and Acoustic Speech Understanding](https://mlsanity.com/paper/2609.22771/) (2026)

## Abstract (arXiv)

This study introduces a method to design a curriculum for machine-learning to maximize the efficiency during the training process of deep neural networks (DNNs) for speech emotion recognition. Previous studies in other machine-learning problems have shown the benefits of training a classifier following a curriculum where samples are gradually presented in increasing level of difficulty. For speech emotion recognition, the challenge is to establish a natural order of difficulty in the training set to create the curriculum. We address this problem by assuming that ambiguous samples for humans are also ambiguous for computers. Speech samples are often annotated by multiple evaluators to account for differences in emotion perception across individuals. While some sentences with clear emotional content are consistently annotated, sentences with more ambiguous emotional content present important disagreement between individual evaluations. We propose to use the disagreement between evaluators as a measure of difficulty for the classification task. We propose metrics that quantify the inter-evaluation agreement to define the curriculum for regression problems and binary and multi-class classification problems. The experimental results consistently show that relying on a curriculum based on agreement between human judgments leads to statistically significant improvements over baselines trained without a curriculum.