# Few-shot Learning in Emotion Recognition of Spontaneous Speech Using a Siamese Neural Network with Adaptive Sample Pair Formation

arXiv: https://arxiv.org/abs/2109.02915 · HTML: https://mlsanity.com/paper/2109.02915/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 51 references that are in the map.
- First posted on arXiv 7 Sep 2021.

## Builds on (most cited first, 15 of 15)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Adversarial Discriminative Domain Adaptation](https://mlsanity.com/paper/1702.05464/) (2017)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [Few-Shot Learning with Metric-Agnostic Conditional Embeddings](https://mlsanity.com/paper/1802.04376/) (2018)
- [Latent Relational Metric Learning via Memory-based Attention for Collaborative Ranking](https://mlsanity.com/paper/1707.05176/) (2017)
- [Cross-view Asymmetric Metric Learning for Unsupervised Person Re-identification](https://mlsanity.com/paper/1708.08062/) (2017)
- [Domain Adversarial for Acoustic Emotion Recognition](https://mlsanity.com/paper/1804.07690/) (2018)
- [Improving Cross-Corpus Speech Emotion Recognition with Adversarial Discriminative Domain Generalization (ADDoG)](https://mlsanity.com/paper/1903.12094/) (2019)
- [Progressive Neural Networks for Transfer Learning in Emotion Recognition](https://mlsanity.com/paper/1706.03256/) (2017)
- [Speech Emotion Recognition via Contrastive Loss under Siamese Networks](https://mlsanity.com/paper/1910.11174/) (2019)
- [Adversarial Auto-encoders for Speech Based Emotion Recognition](https://mlsanity.com/paper/1806.02146/) (2018)
- [A Siamese Neural Network with Modified Distance Loss For Transfer Learning in Speech Emotion Recognition](https://mlsanity.com/paper/2006.03001/) (2020)
- [Learning Representations of Emotional Speech with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1705.02394/) (2017)
- [Transferable Positive/Negative Speech Emotion Recognition via Class-wise Adversarial Domain Adaptation](https://mlsanity.com/paper/1810.12782/) (2018)
- [Learning to Augment Expressions for Few-shot Fine-grained Facial Expression Recognition](https://mlsanity.com/paper/2001.06144/) (2020)

## Built on by (most cited first, 5 of 5)

- [MMAD: Multi-label Micro-Action Detection in Videos](https://mlsanity.com/paper/2407.05311/) (2024)
- [Efficient Labelling of Affective Video Datasets via Few-Shot & Multi-Task Contrastive Learning](https://mlsanity.com/paper/2308.02173/) (2023)
- [A few-shot learning approach with domain adaptation for personalized real-life stress detection in close relationships](https://mlsanity.com/paper/2210.15247/) (2022)
- [A Cross-Lingual Meta-Learning Method Based on Domain Adaptation for Speech Emotion Recognition](https://mlsanity.com/paper/2410.04633/) (2024)
- [HCFSLN: Adaptive Hyperbolic Few-Shot Learning for Multimodal Anxiety Detection](https://mlsanity.com/paper/2511.06988/) (2025)

## Abstract (arXiv)

Speech-based machine learning (ML) has been heralded as a promising solution for tracking prosodic and spectrotemporal patterns in real-life that are indicative of emotional changes, providing a valuable window into one's cognitive and mental state. Yet, the scarcity of labelled data in ambulatory studies prevents the reliable training of ML models, which usually rely on "data-hungry" distribution-based learning. Leveraging the abundance of labelled speech data from acted emotions, this paper proposes a few-shot learning approach for automatically recognizing emotion in spontaneous speech from a small number of labelled samples. Few-shot learning is implemented via a metric learning approach through a siamese neural network, which models the relative distance between samples rather than relying on learning absolute patterns of the corresponding distributions of each emotion. Results indicate the feasibility of the proposed metric learning in recognizing emotions from spontaneous speech in four datasets, even with a small amount of labelled samples. They further demonstrate superior performance of the proposed metric learning compared to commonly used adaptation methods, including network fine-tuning and adversarial learning. Findings from this work provide a foundation for the ambulatory tracking of human emotion in spontaneous speech contributing to the real-life assessment of mental health degradation.