mlsanityData to 7 Oct 2026

Iterative Pseudo-Labeling for Speech Recognition

arXiv 2005.09267 ↗First posted 19 May 2020cs.CL cs.SD eess.AScited by 81

Not on the map. Why?

Abstract (arXiv)

Pseudo-labeling has recently shown promise in end-to-end automatic speech recognition (ASR). We study Iterative Pseudo-Labeling (IPL), a semi-supervised algorithm which efficiently performs multiple iterations of pseudo-labeling on unlabeled data as the acoustic model evolves. In particular, IPL fine-tunes an existing model at each iteration using both labeled data and a subset of unlabeled data. We study the main components of IPL: decoding with a language model and data augmentation. We then demonstrate the effectiveness of IPL by achieving state-of-the-art word-error rate on the Librispeech test sets in both standard and low-resource setting. We also study the effect of language models trained on different corpora to show IPL can effectively utilize additional text. Finally, we release a new large in-domain text corpus which does not overlap with the Librispeech training transcriptions to foster research in low-resource, semi-supervised ASR

Key facts. Cited by 81 papers in the map (data through 7 Oct 2026). First posted on arXiv 19 May 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 21; 2022: 22; 2023: 13; 2024: 11; 2025: 6; 2026: 8.

81 papers.

Sort:

  1. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  2. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  3. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 718
  4. 2022Self-Supervised Speech Representation Learning: A Reviewcited by 258
  5. 2023Google USM: Scaling Automatic Speech Recognition Beyond 100 Languagescited by 228
  6. 2021Unsupervised Speech Recognitioncited by 147
  7. 2021BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognitioncited by 88
  8. 2021Simple and Effective Zero-shot Cross-lingual Phoneme Recognitioncited by 79
  9. 2021Performance-Efficiency Trade-offs in Unsupervised Pre-training for Speech Recognitioncited by 68
  10. 2022Unified Speech-Text Pre-training for Speech Translation and Recognitioncited by 64
  11. 2021Dynamic Distillation Network for Cross-Domain Few-Shot Recognition with Unlabeled Datacited by 49
  12. 2021Towards Building ASR Systems for the Next Billion Userscited by 45
38 more
  1. 2022Towards End-to-end Unsupervised Speech Recognitioncited by 44
  2. 2024A Large-Scale Evaluation of Speech Foundation Modelscited by 38
  3. 2021Momentum Pseudo-Labeling for Semi-Supervised Speech Recognitioncited by 37
  4. 2021Large-Scale Self- and Semi-Supervised Learning for Speech Translationcited by 36
  5. 2024GigaSpeech 2: An Evolving, Large-Scale and Multi-domain ASR Corpus for Low-Resource Languages with Automated Crawling, Transcription and Refinementcited by 34
  6. 2023Making More of Little Data: Improving Low-Resource Automatic Speech Recognition Using Data Augmentationcited by 30
  7. 2021Pseudo-Labeling for Massively Multilingual Speech Recognitioncited by 23
  8. 2023How to Scale Your EMAcited by 22
  9. 2021Kaizen: Continuously improving teacher using Exponential Moving Average for semi-supervised speech recognitioncited by 21
  10. 2021Improved Language Identification Through Cross-Lingual Self-Supervised Learningcited by 20
  11. 2022RemixIT: Continual self-training of speech enhancement models via bootstrapped remixingcited by 20
  12. 2022SPIRAL: Self-supervised Perturbation-Invariant Representation Learning for Speech Pre-Trainingcited by 19
  13. 2022Supervision-Guided Codebooks for Masked Prediction in Speech Pre-trainingcited by 14
  14. 2024Unified Speech Recognition: A Single Model for Auditory, Visual, and Audiovisual Inputscited by 14
  15. 2021Magic dust for cross-lingual adaptation of monolingual wav2vec-2.0cited by 12
  16. 2021On the Use of External Data for Spoken Named Entity Recognitioncited by 12
  17. 2021Contrastive Semi-supervised Learning for ASRcited by 11
  18. 2022Continuous Pseudo-Labeling from the Startcited by 11
  19. 2021The History of Speech Recognition to the Year 2030cited by 10
  20. 2022Improving Mispronunciation Detection with Wav2vec2-based Momentum Pseudo-Labeling for Accentedness and Intelligibility Assessmentcited by 10
  21. 2022Boosting Cross-Domain Speech Recognition with Self-Supervisioncited by 10
  22. 2023LOCATE: Self-supervised Object Discovery via Flow-guided Graph-cut and Bootstrapped Self-trainingcited by 8
  23. 2023A Novel Self-training Approach for Low-resource Speech Recognitioncited by 7
  24. 2021Continual self-training with bootstrapped remixing for speech enhancementcited by 6
  25. 2022Decoupled Federated Learning for ASR with Non-IID Datacited by 6
  26. 2022Make More of Your Data: Minimal Effort Data Augmentation for Automatic Speech Recognition and Translationcited by 6
  27. 2022Continuous Soft Pseudo-Labeling in ASRcited by 6
  28. 2023Alternative Pseudo-Labeling for Semi-Supervised Automatic Speech Recognitioncited by 6
  29. 2022Unsupervised domain adaptation for speech recognition with unsupervised error correctioncited by 5
  30. 2022EURO: ESPnet Unsupervised ASR Open-source Toolkitcited by 5
  31. 2023AV-CPL: Continuous Pseudo-Labeling for Audio-Visual Speech Recognitioncited by 5
  32. 2024Revisiting ASR Error Correction with Specialized Modelscited by 5
  33. 2024Leave No Knowledge Behind During Knowledge Distillation: Towards Practical and Effective Knowledge Distillation for Code-Switching ASR Using Realistic Datacited by 5
  34. 2025Better Pseudo-labeling with Multi-ASR Fusion and Error Correction by SpeechLLMcited by 5
  35. 2022Filter and evolve: progressive pseudo label refining for semi-supervised automatic speech recognitioncited by 4
  36. 2023Pseudo-Labeling for Domain-Agnostic Bangla Automatic Speech Recognitioncited by 4
  37. 2024Semi-Supervised Object Detection: A Survey on Progress from CNN to Transformercited by 4
  38. 2024Empowering Low-Resource Language ASR via Large-Scale Pseudo Labelingcited by 4

50 most cited of 81 All 81 (JSON)