# More Similar than Dissimilar: Modeling Annotators for Cross-Corpus Speech Emotion Recognition

arXiv: https://arxiv.org/abs/2509.12295 · HTML: https://mlsanity.com/paper/2509.12295/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 32 references that are in the map.
- First posted on arXiv 15 Sep 2025.

## Builds on (most cited first, 14 of 14)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotations](https://mlsanity.com/paper/2110.05719/) (2021)
- [Dawn of the transformer era in speech emotion recognition: closing the valence gap](https://mlsanity.com/paper/2203.07378/) (2022)
- [Domain Adversarial for Acoustic Emotion Recognition](https://mlsanity.com/paper/1804.07690/) (2018)
- [Improving Cross-Corpus Speech Emotion Recognition with Adversarial Discriminative Domain Generalization (ADDoG)](https://mlsanity.com/paper/1903.12094/) (2019)
- [PEFT-SER: On the Use of Parameter Efficient Transfer Learning Approaches For Speech Emotion Recognition Using Pre-trained Speech Models](https://mlsanity.com/paper/2306.05350/) (2023)
- [Estimating the Uncertainty in Emotion Attributes using Deep Evidential Regression](https://mlsanity.com/paper/2306.06760/) (2023)
- [MuSE-ing on the Impact of Utterance Ordering On Crowdsourced Emotion Annotations](https://mlsanity.com/paper/1903.11672/) (2019)
- [A study on cross-corpus speech emotion recognition and data augmentation](https://mlsanity.com/paper/2201.03511/) (2022)
- [Pre-Finetuning for Few-Shot Emotional Speech Recognition](https://mlsanity.com/paper/2302.12921/) (2023)
- [The Whole Is Bigger Than the Sum of Its Parts: Modeling Individual Annotators to Capture Emotional Variability](https://mlsanity.com/paper/2408.11956/) (2024)
- [Efficient Finetuning for Dimensional Speech Emotion Recognition in the Age of Transformers](https://mlsanity.com/paper/2503.03756/) (2025)

## Built on by (most cited first, 2 of 2)

- [ADEPT: RL-Aligned Agentic Decoding of Emotion via Evidence Probing Tools -- From Consensus Learning to Ambiguity-Driven Emotion Reasoning](https://mlsanity.com/paper/2602.12714/) (2026)
- [Learning Who Disagrees: Demographic Importance Weighting for Modeling Annotator Distributions with DiADEM](https://mlsanity.com/paper/2604.08425/) (2026)

## Abstract (arXiv)

Speech emotion recognition systems often predict a consensus value generated from the ratings of multiple annotators. However, these models have limited ability to predict the annotation of any one person. Alternatively, models can learn to predict the annotations of all annotators. Adapting such models to new annotators is difficult as new annotators must individually provide sufficient labeled training data. We propose to leverage inter-annotator similarity by using a model pre-trained on a large annotator population to identify a similar, previously seen annotator. Given a new, previously unseen, annotator and limited enrollment data, we can make predictions for a similar annotator, enabling off-the-shelf annotation of unseen data in target datasets, providing a mechanism for extremely low-cost personalization. We demonstrate our approach significantly outperforms other off-the-shelf approaches, paving the way for lightweight emotion adaptation, practical for real-world deployment.