mlsanityData to 7 Oct 2026

Efficient Adapter Transfer of Self-Supervised Speech Models for Automatic Speech Recognition

arXiv 2202.03218 ↗First posted 7 Feb 2022cs.CL cs.SD eess.AScited by 31

Not on the map. Why?

Abstract (arXiv)

Self-supervised learning (SSL) is a powerful tool that allows learning of underlying representations from unlabeled data. Transformer based models such as wav2vec 2.0 and HuBERT are leading the field in the speech domain. Generally these models are fine-tuned on a small amount of labeled data for a downstream task such as Automatic Speech Recognition (ASR). This involves re-training the majority of the model for each task. Adapters are small lightweight modules which are commonly used in Natural Language Processing (NLP) to adapt pre-trained models to new tasks. In this paper we propose applying adapters to wav2vec 2.0 to reduce the number of parameters required for downstream ASR tasks, and increase scalability of the model to multiple tasks or languages. Using adapters we can perform ASR while training fewer than 10% of parameters per task compared to full fine-tuning with little degradation of performance. Ablations show that applying adapters into just the top few layers of the pre-trained network gives similar performance to full transfer, supporting the theory that higher pre-trained layers encode more phonemic information, and further optimizing efficiency.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Builds on 17 of its 20 references that are in the map. First posted on arXiv 7 Feb 2022.

Builds on

17 of 20 references.

References · 17

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  3. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  4. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  5. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  6. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  7. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  8. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,358
  9. 2017Learning multiple visual domains with residual adapterscited by 553
  10. 2019Libri-Light: A Benchmark for ASR with Limited or No Supervisioncited by 462
  11. 2020MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfercited by 443
  12. 2019Simple, Scalable Adaptation for Neural Machine Translationcited by 239
5 more
  1. 2019An Unsupervised Autoregressive Model for Speech Representation Learningcited by 185
  2. 2020Self-training and Pre-training are Complementary for Speech Recognitioncited by 98
  3. 2019Large-Scale Multilingual Speech Recognition with a Streaming End-to-End Modelcited by 92
  4. 2021Lightweight Adapter Tuning for Multilingual Speech Translationcited by 53
  5. 2020Adapt-and-Adjust: Overcoming the Long-Tail Problem of Multilingual Speech Recognitioncited by 34

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 5; 2023: 13; 2024: 8; 2025: 4; 2026: 1.

31 papers.

Sort:

By area:

  1. 2022Self-Supervised Speech Representation Learning: A Reviewcited by 258
  2. 2023Modular Deep Learningcited by 82
  3. 2023Review of Large Vision Models and Visual Prompt Engineeringcited by 77
  4. 2023Deep Transfer Learning for Automatic Speech Recognition: Towards Better Generalizationcited by 28
  5. 2022Exploring Efficient-tuning Methods in Self-supervised Speech Modelscited by 26
  6. 2023Dynamics Generalisation in Reinforcement Learning via Adaptive Context-Aware Policiescited by 18
  7. 2023Multilingual DistilWhisper: Efficient Distillation of Multi-task Speech Models via Language-Specific Expertscited by 15
  8. 2022CHAPTER: Exploiting Convolutional Neural Network Adapters for Self-supervised Speech Modelscited by 13
  9. 2023How to Estimate Model Transferability of Pre-Trained Speech Models?cited by 10
  10. 2023MultiWay-Adapater: Adapting large-scale multi-modal models for scalable image-text retrievalcited by 9
  11. 2023Efficient Computation Sharing for Multi-Task Visual Scene Understandingcited by 8
  12. 2025Accelerating Autoregressive Speech Synthesis Inference With Speech Speculative Decodingcited by 6
19 more
  1. 2023Evaluating Parameter-Efficient Transfer Learning Approaches on SURE Benchmark for Speech Understandingcited by 5
  2. 2024How to Learn a New Language? An Efficient Solution for Self-Supervised Learning Models Unseen Languages Adaption in Low-Resource Scenariocited by 5
  3. 2024Exploration of Adapter for Noise Robust Automatic Speech Recognitioncited by 4
  4. 2023Leveraging Pretrained ASR Encoders for Effective and Efficient End-to-End Speech Intent Classification and Slot Fillingcited by 3
  5. 2023Audio-visual fine-tuning of audio-only ASR modelscited by 3
  6. 2024Efficient Adapter Tuning of Pre-trained Speech Models for Automatic Speaker Verificationcited by 3
  7. 2024ELP-Adapters: Parameter Efficient Adapter Tuning for Various Speech Processing Taskscited by 3
  8. 2024Efficient Compression of Multitask Multilingual Speech Modelscited by 2
  9. 2024Towards scalable efficient on-device ASR with transfer learningcited by 2
  10. 2022Representative Subset Selection for Efficient Fine-Tuning in Self-Supervised Speech Recognitioncited by 1
  11. 2022Damage Control During Domain Adaptation for Transducer Based Automatic Speech Recognitioncited by 1
  12. 2024Dual-Pipeline with Low-Rank Adaptation for New Language Integration in Multilingual ASRcited by 1
  13. 2025UniPET-SPK: A Unified Framework for Parameter-Efficient Tuning of Pre-trained Speech Models for Robust Speaker Verificationcited by 1
  14. 2023Audio-AdapterFusion: A Task-ID-free Approach for Efficient and Non-Destructive Multi-task Speech Recognitioncited by 0
  15. 2023Cascaded Multi-task Adaptive Learning Based on Neural Architecture Searchcited by 0
  16. 2024An Adapter-Based Unified Model for Multiple Spoken Language Processing Taskscited by 0
  17. 2025On-the-fly Routing for Zero-shot MoE Speaker Adaptation of Speech Foundation Models for Dysarthric Speech Recognitioncited by 0
  18. 2025Dialect Identification Using Resource-Efficient Fine-Tuning Approachescited by 0
  19. 2026MambAdapter: Lightweight Mamba-Based Adapters for Parameter-Efficient Transfer Learning in Speech and Audiocited by 0