# Learning Annotation Consensus for Continuous Emotion Recognition

arXiv: https://arxiv.org/abs/2505.21196 · HTML: https://mlsanity.com/paper/2505.21196/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 20 references that are in the map.
- First posted on arXiv 27 May 2025.

## Builds on (most cited first, 6 of 6)

- [Dawn of the transformer era in speech emotion recognition: closing the valence gap](https://mlsanity.com/paper/2203.07378/) (2022)
- [End-to-End Multimodal Emotion Recognition using Deep Neural Networks](https://mlsanity.com/paper/1704.08619/) (2017)
- [Multi-modal Attention for Speech Emotion Recognition](https://mlsanity.com/paper/2009.04107/) (2020)
- [Multistage linguistic conditioning of convolutional layers for speech emotion recognition](https://mlsanity.com/paper/2110.06650/) (2021)
- [Use of Affective Visual Information for Summarization of Human-Centric Videos](https://mlsanity.com/paper/2107.03783/) (2021)
- [Recognizing Emotions evoked by Movies using Multitask Learning](https://mlsanity.com/paper/2107.14529/) (2021)

## Built on by (most cited first, 2 of 2)

- [Resource-Constrained Affect Modelling via Variance Regularisation Pruning](https://mlsanity.com/paper/2605.27479/) (2026)
- [Learning from Annotation Uncertainty: Entropy-Aware Curriculum for Speech Emotion Recognition](https://mlsanity.com/paper/2606.27536/) (2026)

## Abstract (arXiv)

In affective computing, datasets often contain multiple annotations from different annotators, which may lack full agreement. Typically, these annotations are merged into a single gold standard label, potentially losing valuable inter-rater variability. We propose a multi-annotator training approach for continuous emotion recognition (CER) that seeks a consensus across all annotators rather than relying on a single reference label. Our method employs a consensus network to aggregate annotations into a unified representation, guiding the main arousal-valence predictor to better reflect collective inputs. Tested on the RECOLA and COGNIMUSE datasets, our approach outperforms traditional methods that unify annotations into a single label. This underscores the benefits of fully leveraging multi-annotator data in emotion recognition and highlights its applicability across various fields where annotations are abundant yet inconsistent.