mlsanityData to 7 Oct 2026

Conformer-1: Robust ASR via Large-Scale Semisupervised Bootstrapping

arXiv 2404.07341 ↗First posted 10 Apr 2024eess.AS cs.CL cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

This paper presents Conformer-1, an end-to-end Automatic Speech Recognition (ASR) model trained on an extensive dataset of 570k hours of speech audio data, 91% of which was acquired from publicly available sources. To achieve this, we perform Noisy Student Training after generating pseudo-labels for the unlabeled public data using a strong Conformer RNN-T baseline model. The addition of these pseudo-labeled data results in remarkable improvements in relative Word Error Rate (WER) by 11.5% and 24.3% for our asynchronous and realtime models, respectively. Additionally, the model is more robust to background noise owing to the addition of these data. The results obtained in this study demonstrate that the incorporation of pseudo-labeled publicly available data is a highly effective strategy for improving ASR accuracy and noise robustness.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 21 of its 29 references that are in the map. First posted on arXiv 10 Apr 2024.

Builds on

21 of 29 references.

References · 21

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  4. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  5. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  6. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  7. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  8. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,358
  9. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
  10. 2020Stanza: A Python Natural Language Processing Toolkit for Many Human Languagescited by 719
  11. 2013Speech Recognition with Deep Recurrent Neural Networkscited by 666
  12. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 589
9 more
  1. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 555
  2. 2019Libri-Light: A Benchmark for ASR with Limited or No Supervisioncited by 462
  3. 2015End-to-End Attention-based Large Vocabulary Speech Recognitioncited by 148
  4. 2020Improved Noisy Student Training for Automatic Speech Recognitioncited by 139
  5. 2019Self-Training for End-to-End Speech Recognitioncited by 128
  6. 2020Developing Real-time Streaming Transformer Transducer for Speech Recognition on Large-scale Datasetcited by 94
  7. 2020Fast WordPiece Tokenizationcited by 85
  8. 2021Efficient conformer: Progressive downsampling and grouped attention for automatic speech recognitioncited by 42
  9. 2021Momentum Pseudo-Labeling for Semi-Supervised Speech Recognitioncited by 37

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 2.

4 papers.

  1. 2024Anatomy of Industrial Scale Multilingual ASRcited by 14
  2. 2024TouchTTS: An Embarrassingly Simple TTS Framework that Everyone Can Touchcited by 10
  3. 2025Loquacious Set: 25,000 Hours of Transcribed and Diverse English Speech Recognition Data for Research and Commercial Usecited by 9
  4. 2025Pseudo2Real: Task Arithmetic for Pseudo-Label Correction in Automatic Speech Recognitioncited by 1