mlsanityData to 7 Oct 2026

Active Learning with Task Adaptation Pre-training for Speech Emotion Recognition

arXiv 2405.00307 ↗First posted 1 May 2024cs.SD cs.AI eess.AScited by 0

Not on the map. Why?

Abstract (arXiv)

Speech emotion recognition (SER) has garnered increasing attention due to its wide range of applications in various fields, including human-machine interaction, virtual assistants, and mental health assistance. However, existing SER methods often overlook the information gap between the pre-training speech recognition task and the downstream SER task, resulting in sub-optimal performance. Moreover, current methods require much time for fine-tuning on each specific speech dataset, such as IEMOCAP, which limits their effectiveness in real-world scenarios with large-scale noisy data. To address these issues, we propose an active learning (AL)-based fine-tuning framework for SER, called textscAfter, that leverages task adaptation pre-training (TAPT) and AL methods to enhance performance and efficiency. Specifically, we first use TAPT to minimize the information gap between the pre-training speech recognition task and the downstream speech emotion recognition task. Then, AL methods are employed to iteratively select a subset of the most informative and diverse samples for fine-tuning, thereby reducing time consumption. Experiments demonstrate that our proposed method textscAfter, using only 20% of samples, improves accuracy by 8.45% and reduces time consumption by 79%. The additional extension of textscAfter and ablation studies further confirm its effectiveness and applicability to various real-world scenarios. Our source code is available on Github for reproducibility. (https://github.com/Clearloveyuan/AFTER).

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 59 of its 120 references that are in the map. First posted on arXiv 1 May 2024.

Builds on

59 of 120 references.

References · 59

  1. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  3. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  4. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  5. 2020Graph Contrastive Learning with Augmentationscited by 1,203
  6. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 718
  7. 2022Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentationcited by 707
  8. 2019Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Boundscited by 602
  9. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 589
  10. 2019BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learningcited by 428
  11. 2021Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotationscited by 306
  12. 2019Human uncertainty makes classification more robustcited by 231
38 more
  1. 2021Emotion Recognition from Speech Using Wav2vec 2.0 Embeddingscited by 158
  2. 2022Dawn of the transformer era in speech emotion recognition: closing the valence gapcited by 155
  3. 2021Active Learning by Acquiring Contrastive Examplescited by 154
  4. 2020Cold-start Active Learning through Self-supervised Language Modelingcited by 128
  5. 2017Deep learning from crowdscited by 105
  6. 2018The Emotional Voices Database: Towards Controlling the Emotion Dimension in Voice Generation Systemscited by 61
  7. 2022Speech Emotion Recognition using Self-Supervised Featurescited by 52
  8. 2021Exploring Wav2vec 2.0 fine-tuning for improved speech emotion recognitioncited by 47
  9. 2022A Survey of Active Learning for Natural Language Processingcited by 40
  10. 2021Efficient Domain Adaptation of Language Models via Adaptive Tokenizationcited by 34
  11. 2023DPHuBERT: Joint Distillation and Pruning of Self-Supervised Speech Modelscited by 32
  12. 2022In Defense of Core-set: A Density-aware Core-set Selection for Active Learningcited by 26
  13. 2024Residual-based Language Models are Free Boosters for Biomedical Imagingcited by 23
  14. 2024Adaptive Ensembles of Fine-Tuned Transformers for LLM-Generated Text Detectioncited by 22
  15. 2019ShEMO -- A Large-Scale Validated Database for Persian Speech Emotion Detectioncited by 21
  16. 2024A Comparative Study on Enhancing Prediction in Social Network Advertisement through Data Augmentationcited by 20
  17. 2022Temporal Modeling Matters: A Novel Temporal Emotional Modeling Approach for Speech Emotion Recognitioncited by 19
  18. 2021LSSED: a large-scale dataset and benchmark for speech emotion recognitioncited by 17
  19. 2021Representation learning through cross-modal conditional teacher-student training for speech emotion recognitioncited by 17
  20. 2022Multi-task Active Learning for Pre-trained Transformer-based Modelscited by 16
  21. 2023Vesper: A Compact and Effective Pretrained Model for Speech Emotion Recognitioncited by 16
  22. 2022Speech Emotion Recognition using Supervised Deep Recurrent System for Mental Health Monitoringcited by 15
  23. 2023Joyful: Joint Modality Fusion and Graph Contrastive Learning for Multimodal Emotion Recognitioncited by 14
  24. 2021Improved Speech Emotion Recognition using Transfer Learning and Spectrogram Augmentationcited by 12
  25. 2024The New Agronomists: Language Models are Experts in Crop Managementcited by 12
  26. 2020Reducing Confusion in Active Learning for Part-Of-Speech Taggingcited by 11
  27. 2023DST: Deformable Speech Transformer for Emotion Recognitioncited by 11
  28. 2024InsectMamba: Insect Pest Classification with State Space Modelcited by 11
  29. 2021Arabic Speech Emotion Recognition Employing Wav2vec2.0 and HuBERT Based on BAVED Datasetcited by 9
  30. 2022Sentiment-Aware Automatic Speech Recognition pre-training for enhanced Speech Emotion Recognitioncited by 8
  31. 2022CTL-MTNet: A Novel CapsNet and Transfer Learning-Based Mixed Task Net for the Single-Corpus and Cross-Corpus Speech Emotion Recognitioncited by 8
  32. 2020Pre-trained Language Model Based Active Learning for Sentence Matchingcited by 7
  33. 2020Recognizing More Emotions with Less Data Using Self-supervised Transfer Learningcited by 7
  34. 2022Learning Speech Emotion Representations in the Quaternion Domaincited by 7
  35. 2022Fast Yet Effective Speech Emotion Recognition with Self-distillationcited by 7
  36. 2022GM-TCNet: Gated Multi-scale Temporal Convolutional Network using Emotion Causality for Speech Emotion Recognitioncited by 6
  37. 2024ECNet: Effective Controllable Text-to-Image Diffusion Modelscited by 6
  38. 2021Task-adaptive Pre-training of Language Models with Word Embedding Regularizationcited by 5

50 most cited of 59 All 59 (JSON)

Built on by

Not cited yet.