# Parameter Efficient Finetuning for Speech Emotion Recognition and Domain Adaptation

arXiv: https://arxiv.org/abs/2402.11747 · HTML: https://mlsanity.com/paper/2402.11747/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 28 references that are in the map.
- First posted on arXiv 19 Feb 2024.

## Builds on (most cited first, 16 of 16)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models](https://mlsanity.com/paper/2106.10199/) (2021)
- [SUPERB: Speech processing Universal PERformance Benchmark](https://mlsanity.com/paper/2105.01051/) (2021)
- [Self-Supervised Speech Representation Learning: A Review](https://mlsanity.com/paper/2205.10643/) (2022)
- [Emotional Voice Conversion: Theory, Databases and ESD](https://mlsanity.com/paper/2105.14762/) (2021)
- [BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognition](https://mlsanity.com/paper/2109.13226/) (2021)
- [Speech Emotion Recognition using Self-Supervised Features](https://mlsanity.com/paper/2202.03896/) (2022)
- [Emotion recognition by fusing time synchronous and time asynchronous representations](https://mlsanity.com/paper/2010.14102/) (2020)
- [Exploring Efficient-tuning Methods in Self-supervised Speech Models](https://mlsanity.com/paper/2210.06175/) (2022)
- [Self-supervised representations in speech-based depression detection](https://mlsanity.com/paper/2305.12263/) (2023)
- [Representation learning through cross-modal conditional teacher-student training for speech emotion recognition](https://mlsanity.com/paper/2112.00158/) (2021)
- [Speaker Normalization for Self-supervised Speech Emotion Recognition](https://mlsanity.com/paper/2202.01252/) (2022)
- [Evaluating Parameter-Efficient Transfer Learning Approaches on SURE Benchmark for Speech Understanding](https://mlsanity.com/paper/2303.03267/) (2023)

## Built on by (most cited first, 3 of 3)

- [Cross-Lingual Speech Emotion Recognition: Humans vs. Self-Supervised Models](https://mlsanity.com/paper/2409.16920/) (2024)
- [Exploring Acoustic Similarity in Emotional Speech and Music via Self-Supervised Representations](https://mlsanity.com/paper/2409.17899/) (2024)
- [EmotionAI: A Privacy-Preserving Computational Intelligence Pipeline for Speech-Emotion-Grounded Conversational Analysis](https://mlsanity.com/paper/2606.24941/) (2026)

## Abstract (arXiv)

Foundation models have shown superior performance for speech emotion recognition (SER). However, given the limited data in emotion corpora, finetuning all parameters of large pre-trained models for SER can be both resource-intensive and susceptible to overfitting. This paper investigates parameter-efficient finetuning (PEFT) for SER. Various PEFT adaptors are systematically studied for both classification of discrete emotion categories and prediction of dimensional emotional attributes. The results demonstrate that the combination of PEFT methods surpasses full finetuning with a significant reduction in the number of trainable parameters. Furthermore, a two-stage adaptation strategy is proposed to adapt models trained on acted emotion data, which is more readily available, to make the model more adept at capturing natural emotional expressions. Both intra- and cross-corpus experiments validate the efficacy of the proposed approach in enhancing the performance on both the source and target domains.