# Exploring speaker enrolment for few-shot personalisation in emotional vocalisation prediction

arXiv: https://arxiv.org/abs/2206.06680 · HTML: https://mlsanity.com/paper/2206.06680/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 22 references that are in the map.
- First posted on arXiv 14 Jun 2022.

## Builds on (most cited first, 10 of 10)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition](https://mlsanity.com/paper/1912.10211/) (2019)
- [End-to-End Multimodal Emotion Recognition using Deep Neural Networks](https://mlsanity.com/paper/1704.08619/) (2017)
- [Personalized Machine Learning for Robot Perception of Affect and Engagement in Autism Therapy](https://mlsanity.com/paper/1802.01186/) (2018)
- [The ICML 2022 Expressive Vocalizations Workshop and Competition: Recognizing, Generating, and Personalizing Vocal Bursts](https://mlsanity.com/paper/2205.01780/) (2022)
- [Speaker-invariant Affective Representation Learning via Adversarial Training](https://mlsanity.com/paper/1911.01533/) (2019)
- [Speaker Attentive Speech Emotion Recognition](https://mlsanity.com/paper/2104.07288/) (2021)
- [Unsupervised Personalization of an Emotion Recognition System: The Unique Properties of the Externalization of Valence in Speech](https://mlsanity.com/paper/2201.07876/) (2022)

## Built on by (most cited first, 3 of 3)

- [Beyond One-Size-Fits-All: A Survey of Personalized Affective Computing in Human-Agent Interaction](https://mlsanity.com/paper/2304.00377/) (2023)
- [Enrolment-based personalisation for improving individual-level fairness in speech emotion recognition](https://mlsanity.com/paper/2406.06665/) (2024)
- [Self-Supervised Attention Networks and Uncertainty Loss Weighting for Multi-Task Emotion Recognition on Vocal Bursts](https://mlsanity.com/paper/2209.07384/) (2022)

## Abstract (arXiv)

In this work, we explore a novel few-shot personalisation architecture for emotional vocalisation prediction. The core contribution is an `enrolment' encoder which utilises two unlabelled samples of the target speaker to adjust the output of the emotion encoder; the adjustment is based on dot-product attention, thus effectively functioning as a form of `soft' feature selection. The emotion and enrolment encoders are based on two standard audio architectures: CNN14 and CNN10. The two encoders are further guided to forget or learn auxiliary emotion and/or speaker information. Our best approach achieves a CCC of .650 on the ExVo Few-Shot dev set, a 2.5% increase over our baseline CNN14 CCC of .634.