mlsanityData to 7 Oct 2026

Exploring Efficient-tuning Methods in Self-supervised Speech Models

arXiv 2210.06175 ↗First posted 10 Oct 2022eess.AS cs.LG cs.SDcited by 26

Not on the map. Why?

Abstract (arXiv)

In this study, we aim to explore efficient tuning methods for speech self-supervised learning. Recent studies show that self-supervised learning (SSL) can learn powerful representations for different speech tasks. However, fine-tuning pre-trained models for each downstream task is parameter-inefficient since SSL models are notoriously large with millions of parameters. Adapters are lightweight modules commonly used in NLP to solve this problem. In downstream tasks, the parameters of SSL models are frozen, and only the adapters are trained. Given the lack of studies generally exploring the effectiveness of adapters for self-supervised speech tasks, we intend to fill this gap by adding various adapter modules in pre-trained speech SSL models. We show that the performance parity can be achieved with over 90% parameter reduction, and discussed the pros and cons of efficient tuning techniques. This is the first comprehensive investigation of various adapter types across speech tasks.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Builds on 22 of its 24 references that are in the map. First posted on arXiv 10 Oct 2022.

Builds on

22 of 24 references.

References · 22

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  4. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  5. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  6. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  7. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,234
  8. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 863
  9. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 652
  10. 2020MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfercited by 443
  11. 2021Generative Spoken Language Modeling from Raw Audiocited by 336
  12. 2021Layer-wise Analysis of a Self-supervised Speech Representation Modelcited by 293
10 more
  1. 2022Self-Supervised Speech Representation Learning: A Reviewcited by 258
  2. 2021On the Effectiveness of Adapter-based Tuning for Pretrained Language Model Adaptationcited by 147
  3. 2021Voice2Series: Reprogramming Acoustic Models for Time Series Classificationcited by 91
  4. 2020DeCoAR 2.0: Deep Contextualized Acoustic Representations with Vector Quantizationcited by 77
  5. 2021Counter-Interference Adapter for Multilingual Machine Translationcited by 39
  6. 2022Efficient Adapter Transfer of Self-Supervised Speech Models for Automatic Speech Recognitioncited by 31
  7. 2022AdapterBias: Parameter-efficient Token-dependent Representation Shift for Adapters in NLP Taskscited by 29
  8. 2022DRAFT: A Novel Framework to Reduce Domain Shifting in Self-supervised Learning and Its Application to Children's ASRcited by 21
  9. 2021An Adapter Based Pre-Training for Efficient and Scalable Self-Supervised Speech Representation Learningcited by 19
  10. 2022SpeechPrompt: An Exploration of Prompt Tuning on Generative Spoken Language Model for Speech Processing Taskscited by 19

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 9; 2024: 11; 2025: 3; 2026: 1.

26 papers.

Sort:

By area:

  1. 2023Dynamic-SUPERB: Towards A Dynamic, Collaborative, and Comprehensive Instruction-Tuning Benchmark for Speechcited by 99
  2. 2023Modular Deep Learningcited by 82
  3. 2023Whispering LLaMA: A Cross-Modal Generative Error Correction Framework for Speech Recognitioncited by 56
  4. 2023Learning from models beyond fine-tuningcited by 47
  5. 2022SUPERB @ SLT 2022: Challenge on Generalization and Efficiency of Self-Supervised Speech Representation Learningcited by 26
  6. 2024ML-SUPERB 2.0: Benchmarking Multilingual Speech Models Across Modeling Constraints, Languages, and Datasetscited by 26
  7. 2024It's Never Too Late: Fusing Acoustic Information into Large Language Models for Automatic Speech Recognitioncited by 22
  8. 2022CHAPTER: Exploiting Convolutional Neural Network Adapters for Self-supervised Speech Modelscited by 13
  9. 2023Low-rank Adaptation of Large Language Model Rescoring for Parameter-Efficient Speech Recognitioncited by 13
  10. 2023How to Estimate Model Transferability of Pre-Trained Speech Models?cited by 10
  11. 2024How to Learn a New Language? An Efficient Solution for Self-Supervised Learning Models Unseen Languages Adaption in Low-Resource Scenariocited by 5
  12. 2025Adapting Whisper for Parameter-efficient Code-Switching Speech Recognition via Soft Prompt Tuningcited by 5
14 more
  1. 2023MERTech: Instrument Playing Technique Detection Using Self-Supervised Pretrained Model With Multi-Task Finetuningcited by 4
  2. 2024Noise-robust zero-shot text-to-speech synthesis conditioned on self-supervised speech-representation model with adapterscited by 4
  3. 2023Prompting and Adapter Tuning for Self-supervised Encoder-Decoder Speech Modelcited by 3
  4. 2024PEFT for Speech: Unveiling Optimal Placement, Merging Strategies, and Ensemble Techniquescited by 3
  5. 2024Parameter Efficient Finetuning for Speech Emotion Recognition and Domain Adaptationcited by 3
  6. 2024ELP-Adapters: Parameter Efficient Adapter Tuning for Various Speech Processing Taskscited by 3
  7. 2024Lightweight Zero-shot Text-to-Speech with Mixture of Adapterscited by 2
  8. 2024CA-SSLR: Condition-Aware Self-Supervised Learning Representation for Generalized Speech Processingcited by 2
  9. 2025Speech-FT: Merging Pre-trained And Fine-Tuned Speech Representation Models For Cross-Task Generalizationcited by 2
  10. 2024Dual-Pipeline with Low-Rank Adaptation for New Language Integration in Multilingual ASRcited by 1
  11. 2023BLIP-Adapter: Parameter-Efficient Transfer Learning for Mobile Screenshot Captioningcited by 0
  12. 2024JOOCI: a Framework for Learning Comprehensive Speech Representationscited by 0
  13. 2025On-the-fly Routing for Zero-shot MoE Speaker Adaptation of Speech Foundation Models for Dysarthric Speech Recognitioncited by 0
  14. 2026Closing the Quality Gap in Low-Resource Text-to-Speech: LoRA Fine-Tuning of VoxCPM2 for Khmer and Koreancited by 0