mlsanityData to 14 Oct 2026

Improving Noisy Student Training on Non-target Domain Data for Automatic Speech Recognition

arXiv 2211.04717 ↗First posted 9 Nov 2022cs.SD cs.CL eess.AScited by 5

Not on the map. Why?

Abstract (arXiv)

Noisy Student Training (NST) has recently demonstrated extremely strong performance in Automatic Speech Recognition(ASR). In this paper, we propose a data selection strategy named LM Filter to improve the performance of NST on non-target domain data in ASR tasks. Hypotheses with and without a Language Model are generated and the CER differences between them are utilized as a filter threshold. Results reveal that significant improvements of 10.4% compared with no data filtering baselines. We can achieve 3.31% CER in AISHELL-1 test set, which is best result from our knowledge without any other supervised data. We also perform evaluations on the supervised 1000 hour AISHELL-2 dataset and competitive results of 4.73% CER can be achieved.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 18 of its 29 references that are in the map. First posted on arXiv 9 Nov 2022.

Builds on

18 of 29 references.

References · 18

  1. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  2. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  3. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,380
  4. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,311
  5. 2017AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baselinecited by 373
  6. 2019Billion-scale semi-supervised learning for image classificationcited by 232
  7. 2021WenetSpeech: A 10000+ Hours Multi-domain Mandarin Corpus for Speech Recognitioncited by 197
  8. 2020Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognitioncited by 187
  9. 2019Revisiting Self-Training for Neural Sequence Generationcited by 155
  10. 2020Improved Noisy Student Training for Automatic Speech Recognitioncited by 141
  11. 2018AISHELL-2: Transforming Mandarin ASR Research Into Industrial Scalecited by 128
  12. 2021WeNet: Production oriented Streaming and Non-streaming End-to-End Speech Recognition Toolkitcited by 125
6 more
  1. 2019End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architecturescited by 120
  2. 2020Improving Streaming Automatic Speech Recognition With Non-Streaming Model Distillation On Unsupervised Datacited by 16
  3. 2021Scaling ASR Improves Zero and Few Shot Learningcited by 16
  4. 2022Unsupervised Data Selection via Discrete Speech Representation for ASRcited by 10
  5. 2019Semi-Supervised Segmentation of Salt Bodies in Seismic Images using an Ensemble of Convolutional Neural Networkscited by 9
  6. 2022FedNST: Federated Noisy Student Training for Automatic Speech Recognitioncited by 5

Built on by

Cited by year

Citing papers per year: 2024: 4; 2025: 1.

5 papers.

  1. 2024GigaSpeech 2: An Evolving, Large-Scale and Multi-domain ASR Corpus for Low-Resource Languages with Automated Crawling, Transcription and Refinementcited by 34
  2. 2024On the Effects of Heterogeneous Data Sources on Speech-to-Text Foundation Modelscited by 10
  3. 2024Empowering Low-Resource Language ASR via Large-Scale Pseudo Labelingcited by 5
  4. 2024Token-Weighted RNN-T for Learning from Flawed Datacited by 2
  5. 2025LESS: Large Language Model Enhanced Semi-Supervised Learning for Speech Foundational Models Using in-the-wild Datacited by 0