# Active Learning Based Fine-Tuning Framework for Speech Emotion Recognition

arXiv: https://arxiv.org/abs/2310.00283 · HTML: https://mlsanity.com/paper/2310.00283/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 50 references that are in the map.
- First posted on arXiv 30 Sep 2023.

## Builds on (most cited first, 25 of 25)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [Graph Contrastive Learning with Augmentations](https://mlsanity.com/paper/2010.13902/) (2020)
- [data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language](https://mlsanity.com/paper/2202.03555/) (2022)
- [XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale](https://mlsanity.com/paper/2111.09296/) (2021)
- [BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning](https://mlsanity.com/paper/1906.08158/) (2019)
- [Dawn of the transformer era in speech emotion recognition: closing the valence gap](https://mlsanity.com/paper/2203.07378/) (2022)
- [Active Learning by Acquiring Contrastive Examples](https://mlsanity.com/paper/2109.03764/) (2021)
- [Cold-start Active Learning through Self-supervised Language Modeling](https://mlsanity.com/paper/2010.09535/) (2020)
- [The Emotional Voices Database: Towards Controlling the Emotion Dimension in Voice Generation Systems](https://mlsanity.com/paper/1806.09514/) (2018)
- [Speech Emotion Recognition using Self-Supervised Features](https://mlsanity.com/paper/2202.03896/) (2022)
- [Exploring Wav2vec 2.0 fine-tuning for improved speech emotion recognition](https://mlsanity.com/paper/2110.06309/) (2021)
- [ShEMO -- A Large-Scale Validated Database for Persian Speech Emotion Detection](https://mlsanity.com/paper/1906.01155/) (2019)
- [LSSED: a large-scale dataset and benchmark for speech emotion recognition](https://mlsanity.com/paper/2102.01754/) (2021)
- [Vesper: A Compact and Effective Pretrained Model for Speech Emotion Recognition](https://mlsanity.com/paper/2307.10757/) (2023)
- [Speech Emotion Recognition using Supervised Deep Recurrent System for Mental Health Monitoring](https://mlsanity.com/paper/2208.12812/) (2022)
- [Improved Speech Emotion Recognition using Transfer Learning and Spectrogram Augmentation](https://mlsanity.com/paper/2108.02510/) (2021)
- [Sentiment-Aware Automatic Speech Recognition pre-training for enhanced Speech Emotion Recognition](https://mlsanity.com/paper/2201.11826/) (2022)
- [Learning Speech Emotion Representations in the Quaternion Domain](https://mlsanity.com/paper/2204.02385/) (2022)
- [Fast Yet Effective Speech Emotion Recognition with Self-distillation](https://mlsanity.com/paper/2210.14636/) (2022)
- [Speech Emotion Recognition with Global-Aware Fusion on Multi-scale Feature Representation](https://mlsanity.com/paper/2204.05571/) (2022)
- [Unsupervised low-rank representations for speech emotion recognition](https://mlsanity.com/paper/2104.07072/) (2021)
- [Domain Specific Wav2vec 2.0 Fine-tuning For The SE&R 2022 Challenge](https://mlsanity.com/paper/2207.14418/) (2022)

## Built on by (most cited first, 2 of 2)

- [A Survey on Deep Active Learning: Recent Advances and New Frontiers](https://mlsanity.com/paper/2405.00334/) (2024)
- [Active Learning with Task Adaptation Pre-training for Speech Emotion Recognition](https://mlsanity.com/paper/2405.00307/) (2024)

## Abstract (arXiv)

Speech emotion recognition (SER) has drawn increasing attention for its applications in human-machine interaction. However, existing SER methods ignore the information gap between the pre-training speech recognition task and the downstream SER task, leading to sub-optimal performance. Moreover, they require much time to fine-tune on each specific speech dataset, restricting their effectiveness in real-world scenes with large-scale noisy data. To address these issues, we propose an active learning (AL) based Fine-Tuning framework for SER that leverages task adaptation pre-training (TAPT) and AL methods to enhance performance and efficiency. Specifically, we first use TAPT to minimize the information gap between the pre-training and the downstream task. Then, AL methods are used to iteratively select a subset of the most informative and diverse samples for fine-tuning, reducing time consumption. Experiments demonstrate that using only 20%pt. samples improves 8.45%pt. accuracy and reduces 79%pt. time consumption.