mlsanityData to 14 Oct 2026

Large-scale ASR Domain Adaptation using Self- and Semi-supervised Learning

arXiv 2110.00165 ↗First posted 1 Oct 2021eess.AS cs.CL cs.LGcited by 26

Not on the map. Why?

Abstract (arXiv)

Self- and semi-supervised learning methods have been actively investigated to reduce labeled training data or enhance the model performance. However, the approach mostly focus on in-domain performance for public datasets. In this study, we utilize the combination of self- and semi-supervised learning methods to solve unseen domain adaptation problem in a large-scale production setting for online ASR model. This approach demonstrates that using the source domain data with a small fraction of the target domain data (3%) can recover the performance gap compared to a full data baseline: relative 13.5% WER improvement for target domain data.

Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). Builds on 15 of its 24 references that are in the map. First posted on arXiv 1 Oct 2021.

Builds on

15 of 24 references.

References · 15

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  3. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  4. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  5. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,311
  6. 2019Libri-Light: A Benchmark for ASR with Limited or No Supervisioncited by 467
  7. 2020Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognitioncited by 187
  8. 2020Improved Noisy Student Training for Automatic Speech Recognitioncited by 141
  9. 2020Multi-task self-supervised learning for Robust Speech Recognitioncited by 130
  10. 2019Generative Pre-Training for Speech with Autoregressive Predictive Codingcited by 82
  11. 2019Recognizing long-form speech using streaming end-to-end modelscited by 77
  12. 2020A Better and Faster End-to-End Model for Streaming ASRcited by 69
3 more
  1. 2018Toward domain-invariant speech recognition via large scale trainingcited by 35
  2. 2021Learning Word-Level Confidence For Subword End-to-End ASRcited by 22
  3. 2021Contrastive Semi-supervised Learning for ASRcited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 2; 2022: 10; 2023: 4; 2024: 3; 2025: 5; 2026: 2.

26 papers.

Sort:

By area:

  1. 2021Joint Unsupervised and Supervised Training for Multilingual ASRcited by 39
  2. 2022Pseudo Label Is Better Than Human Labelcited by 25
  3. 2024Anatomy of Industrial Scale Multilingual ASRcited by 14
  4. 2022Maestro-U: Leveraging joint speech-text representation learning for zero supervised speech ASRcited by 13
  5. 2023Efficient Domain Adaptation for Speech Foundation Modelscited by 13
  6. 2023Low-rank Adaptation of Large Language Model Rescoring for Parameter-Efficient Speech Recognitioncited by 13
  7. 2022PADA: Pruning Assisted Domain Adaptation for Self-Supervised Speech Representationscited by 11
  8. 2022Boosting Cross-Domain Speech Recognition with Self-Supervisioncited by 10
  9. 2022Improving Deliberation by Text-Only and Semi-Supervised Trainingcited by 10
  10. 2022Production federated keyword spotting via distillation, filtering, and joint federated-centralized trainingcited by 9
  11. 2024Self-Taught Recognizer: Toward Unsupervised Adaptation for Speech Foundation Modelscited by 8
  12. 2021Wav2vec-S: Semi-Supervised Pre-Training for Low-Resource ASRcited by 7
14 more
  1. 2022Sample-Efficient Unsupervised Domain Adaptation of Speech Recognition Systems A case study for Modern Greekcited by 7
  2. 2022Large vocabulary speech recognition for languages of Africa: multilingual modeling and self-supervised learningcited by 6
  3. 2024Semi-Self-Supervised Domain Adaptation: Developing Deep Learning Models with Limited Annotated Data for Wheat Head Segmentationcited by 5
  4. 2023Confidence-based Ensembles of End-to-End Speech Recognition Modelscited by 4
  5. 2022Resource-Efficient Transfer Learning From Speech Foundation Model Using Hierarchical Feature Fusioncited by 3
  6. 2025Better Semi-supervised Learning for Multi-domain ASR Through Incremental Retraining and Data Filteringcited by 2
  7. 2025BEST-RQ-Based Self-Supervised Learning for Whisper Domain Adaptationcited by 2
  8. 2022Damage Control During Domain Adaptation for Transducer Based Automatic Speech Recognitioncited by 1
  9. 2025MSDA: Combining Pseudo-labeling and Self-Supervision for Unsupervised Domain Adaptation in ASRcited by 1
  10. 2025Customizing Speech Recognition Model with Large Language Model Feedbackcited by 1
  11. 2026Gumbel-BEARD: Automatic Layer Selection for Self-Supervised Adaptation of Whisper in Low-Resource Domainscited by 1
  12. 2023Feature Normalization for Fine-tuning Self-Supervised Models in Speech Enhancementcited by 0
  13. 2025LESS: Large Language Model Enhanced Semi-Supervised Learning for Speech Foundational Models Using in-the-wild Datacited by 0
  14. 2026Closing the Speech-Text Gap with Limited Audio for Effective Domain Adaptation in LLM-Based ASRcited by 0