# Personalized Adaptation with Pre-trained Speech Encoders for Continuous Emotion Recognition

arXiv: https://arxiv.org/abs/2309.02418 · HTML: https://mlsanity.com/paper/2309.02418/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Sep 2023.

## Built on by (most cited first, 4 of 4)

- [Enhancing Modal Fusion by Alignment and Label Matching for Multimodal Emotion Recognition](https://mlsanity.com/paper/2408.09438/) (2024)
- [Meta-PerSER: Few-Shot Listener Personalized Speech Emotion Recognition via Meta-learning](https://mlsanity.com/paper/2505.16220/) (2025)
- [Retrieval-Augmented Personalization with Foundation Models for Wearable Stress Detection](https://mlsanity.com/paper/2606.24985/) (2026)
- [Test-Time Adaptation for Speech Emotion Recognition](https://mlsanity.com/paper/2601.16240/) (2026)

## Abstract (arXiv)

There are individual differences in expressive behaviors driven by cultural norms and personality. This between-person variation can result in reduced emotion recognition performance. Therefore, personalization is an important step in improving the generalization and robustness of speech emotion recognition. In this paper, to achieve unsupervised personalized emotion recognition, we first pre-train an encoder with learnable speaker embeddings in a self-supervised manner to learn robust speech representations conditioned on speakers. Second, we propose an unsupervised method to compensate for the label distribution shifts by finding similar speakers and leveraging their label distributions from the training set. Extensive experimental results on the MSP-Podcast corpus indicate that our method consistently outperforms strong personalization baselines and achieves state-of-the-art performance for valence estimation.