mlsanityData to 7 Oct 2026

Residual Adapters for Parameter-Efficient ASR Adaptation to Atypical and Accented Speech

arXiv 2109.06952 ↗First posted 14 Sep 2021cs.CL cs.SD eess.AScited by 30

Not on the map. Why?

Abstract (arXiv)

Automatic Speech Recognition (ASR) systems are often optimized to work best for speakers with canonical speech patterns. Unfortunately, these systems perform poorly when tested on atypical speech and heavily accented speech. It has previously been shown that personalization through model fine-tuning substantially improves performance. However, maintaining such large models per speaker is costly and difficult to scale. We show that by adding a relatively small number of extra parameters to the encoder layers via so-called residual adapter, we can achieve similar adaptation gains compared to model fine-tuning, while only updating a tiny fraction (less than 0.5%) of the model parameters. We demonstrate this on two speech adaptation tasks (atypical and accented speech) and for two state-of-the-art ASR architectures.

Key facts. Cited by 30 papers in the map (data through 7 Oct 2026). Builds on 20 of its 30 references that are in the map. First posted on arXiv 14 Sep 2021.

Builds on

20 of 30 references.

References · 20

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  3. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,358
  4. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
  5. 2013Speech Recognition with Deep Recurrent Neural Networkscited by 666
  6. 2017Learning multiple visual domains with residual adapterscited by 553
  7. 2019How to Fine-Tune BERT for Text Classification?cited by 409
  8. 2019Simple, Scalable Adaptation for Neural Machine Translationcited by 239
  9. 2020Transformer Transducer: A Streamable Speech Recognition Model with Transformer Encoders and RNN-T Losscited by 232
  10. 2018Streaming End-to-end Speech Recognition For Mobile Devicescited by 230
  11. 2020A Streaming On-Device End-to-End Model Surpassing Server-Side Conventional Model Quality and Latencycited by 99
  12. 2019Large-Scale Multilingual Speech Recognition with a Streaming End-to-End Modelcited by 92
8 more
  1. 2019Recognizing long-form speech using streaming end-to-end modelscited by 77
  2. 2019Personalizing ASR for Dysarthric and Accented Speech with Limited Datacited by 45
  3. 2021Advancing RNN Transducer Technology for Speech Recognitioncited by 44
  4. 2016Learning Hidden Unit Contributions for Unsupervised Acoustic Model Adaptationcited by 38
  5. 2016Fast Domain Adaptation for Neural Machine Translationcited by 28
  6. 2019Parrotron: An End-to-End Speech-to-Speech Conversion Model and its Applications to Hearing-Impaired Speech and Speech Separationcited by 22
  7. 2020Multi-Accent Adaptation based on Gate Mechanismcited by 9
  8. 2019Leveraging native language information for improved accented speech recognitioncited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 14; 2024: 8; 2025: 2; 2026: 3.

30 papers.

Sort:

By area:

  1. 2023Google USM: Scaling Automatic Speech Recognition Beyond 100 Languagescited by 228
  2. 2024A Survey of Resource-efficient LLM and Multimodal Foundation Modelscited by 84
  3. 2023Modular Deep Learningcited by 82
  4. 2023From English to More Languages: Parameter-Efficient Model Reprogramming for Cross-Lingual Speech Recognitioncited by 23
  5. 2023LatentEditor: Text Driven Local Editing of 3D Scenescited by 17
  6. 2023Multilingual DistilWhisper: Efficient Distillation of Multi-task Speech Models via Language-Specific Expertscited by 15
  7. 2022Residual Adapters for Few-Shot Text-to-Speech Speaker Adaptationcited by 10
  8. 2023Disentangling Voice and Content with Self-Supervision for Speaker Recognitioncited by 10
  9. 2024RST-LoRA: A Discourse-Aware Low-Rank Adaptation for Long Document Abstractive Summarizationcited by 10
  10. 2023Efficient Computation Sharing for Multi-Task Visual Scene Understandingcited by 8
  11. 2023A Parameter-Efficient Learning Approach to Arabic Dialect Identification with Pre-Trained General-Purpose Speech Modelcited by 8
  12. 2022Federated Learning for Inference at Anytime and Anywherecited by 5
18 more
  1. 2023Parameter-Efficient Learning for Text-to-Speech Accent Adaptationcited by 4
  2. 2023Latent Phrase Matching for Dysarthric Speechcited by 2
  3. 2023Don't Stop Self-Supervision: Accent Adaptation of Speech Representations via Residual Adapterscited by 2
  4. 2024Hierarchical Recurrent Adapters for Efficient Multi-Task Adaptation of Large Speech Modelscited by 2
  5. 2024Efficient Compression of Multitask Multilingual Speech Modelscited by 2
  6. 2022Damage Control During Domain Adaptation for Transducer Based Automatic Speech Recognitioncited by 1
  7. 2023Differentially Private Adapters for Parameter Efficient Acoustic Modelingcited by 1
  8. 2024Hypernetworks for Personalizing ASR to Atypical Speechcited by 1
  9. 2024Improving Self-supervised Pre-training using Accent-Specific Codebookscited by 1
  10. 2024UME: Upcycling Mixture-of-Experts for Scalable and Efficient Automatic Speech Recognitioncited by 1
  11. 2025MoFE: Mixture of Frozen Experts Architecturecited by 1
  12. 2025Idiosyncratic Versus Normative Modeling of Atypical Speech Recognition: Dysarthric Case Studiescited by 1
  13. 2023A meta learning scheme for fast accent domain expansion in Mandarin speech recognitioncited by 0
  14. 2023Cascaded Multi-task Adaptive Learning Based on Neural Architecture Searchcited by 0
  15. 2024Parameter-efficient Adaptation of Multilingual Multimodal Models for Low-resource ASRcited by 0
  16. 2026Mixture-of-Experts with Intermediate CTC Supervision for Accented Speech Recognitioncited by 0
  17. 2026ACES: Accent Subspaces for Coupling, Explanations, and Stress-Testing in Automatic Speech Recognitioncited by 0
  18. 2026Huntington Disease Automatic Speech Recognition with Biomarker Supervisioncited by 0