mlsanityData to 7 Oct 2026

SpeechStew: Simply Mix All Available Speech Recognition Data to Train One Large Neural Network

arXiv 2104.02133 ↗First posted 5 Apr 2021cs.CL cs.LGcited by 83

Not on the map. Why?

Abstract (arXiv)

We present SpeechStew, a speech recognition model that is trained on a combination of various publicly available speech recognition datasets: AMI, Broadcast News, Common Voice, LibriSpeech, Switchboard/Fisher, Tedlium, and Wall Street Journal. SpeechStew simply mixes all of these datasets together, without any special re-weighting or re-balancing of the datasets. SpeechStew achieves SoTA or near SoTA results across a variety of tasks, without the use of an external language model. Our results include 9.0% WER on AMI-IHM, 4.7% WER on Switchboard, 8.3% WER on CallHome, and 1.3% on WSJ, which significantly outperforms prior work with strong external language models. We also demonstrate that SpeechStew learns powerful transfer learning representations. We fine-tune SpeechStew on a noisy low resource speech dataset, CHiME-6. We achieve 38.9% WER without a language model, which compares to 38.6% WER to a strong HMM baseline with a language model.

Key facts. Cited by 83 papers in the map (data through 7 Oct 2026). Builds on 36 of its 57 references that are in the map. First posted on arXiv 5 Apr 2021.

Builds on

36 of 57 references.

References · 36

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2020Scaling Laws for Neural Language Modelscited by 6,411
  6. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  7. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  8. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,375
  9. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,358
  10. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
  11. 2018Adafactor: Adaptive Learning Rates with Sublinear Memory Costcited by 896
  12. 2019Big Transfer (BiT): General Visual Representation Learningcited by 766
24 more
  1. 2019Deep Double Descent: Where Bigger Models and More Data Hurtcited by 667
  2. 2018Do Better ImageNet Models Transfer Better?cited by 560
  3. 2015Listen, Attend and Spellcited by 542
  4. 2019Libri-Light: A Benchmark for ASR with Limited or No Supervisioncited by 462
  5. 2013DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognitioncited by 422
  6. 2014Caffe: Convolutional Architecture for Fast Feature Embeddingcited by 287
  7. 2018TED-LIUM 3: twice as much data and corpus repartition for experiments on speaker adaptationcited by 224
  8. 2017State-of-the-art Speech Recognition With Sequence-to-Sequence Modelscited by 196
  9. 2020Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognitioncited by 185
  10. 2020Improved Noisy Student Training for Automatic Speech Recognitioncited by 139
  11. 2020CHiME-6 Challenge:Tackling Multispeaker Speech Recognition for Unsegmented Recordingscited by 129
  12. 2019End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architecturescited by 120
  13. 2020Rethinking Evaluation in ASR: Are Our Models Robust Enough?cited by 72
  14. 2020Imputer: Sequence Modelling via Imputation and Dynamic Programmingcited by 65
  15. 2019SpecAugment on Large Scale Datasetscited by 60
  16. 2018Bytes are All You Need: End-to-End Multilingual Speech Recognition and Synthesis with Bytescited by 54
  17. 2018Toward domain-invariant speech recognition via large scale trainingcited by 35
  18. 2016Very Deep Convolutional Networks for End-to-End Speech Recognitioncited by 34
  19. 2021Large-Scale Pre-Training of End-to-End Multi-Talker ASR for Meeting Transcription with Single Distant Microphonecited by 33
  20. 2020The RWTH ASR System for TED-LIUM Release 2: Improving Hybrid HMM with SpecAugmentcited by 19
  21. 2021Transformer Language Models with LSTM-based Cross-utterance Information Representationcited by 15
  22. 2018Optimal Completion Distillation for Sequence Learningcited by 14
  23. 2020An investigation of phone-based subword units for end-to-end speech recognitioncited by 12
  24. 2020Towards a Competitive End-to-End Speech Recognition for CHiME-6 Dinner Party Transcriptioncited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 13; 2022: 18; 2023: 29; 2024: 12; 2025: 9; 2026: 2.

83 papers.

Sort:

By area:

  1. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  2. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  3. 2023Scaling Speech Technology to 1,000+ Languagescited by 401
  4. 2023Google USM: Scaling Automatic Speech Recognition Beyond 100 Languagescited by 228
  5. 2021BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognitioncited by 88
  6. 2023Distil-Whisper: Robust Knowledge Distillation via Large-Scale Pseudo Labellingcited by 87
  7. 2022MAESTRO: Matched Speech Text Representations through Modality Matchingcited by 79
  8. 2021SLAM: A Unified Encoder for Speech and Language Modeling via Speech-Text Joint Pre-Trainingcited by 69
  9. 2023SLM: Bridge the thin gap between speech and text foundation modelscited by 67
  10. 2022UnitY: Two-pass Direct Speech-to-speech Translation with Discrete Unitscited by 60
  11. 2021PARP: Prune, Adjust and Re-Prune for Self-Supervised Speech Recognitioncited by 50
  12. 2021CAPE: Encoding Relative Positions with Continuous Augmented Positional Embeddingscited by 47
38 more
  1. 2022Enhanced Direct Speech-to-Speech Translation Using Self-supervised Pre-training and Data Augmentationcited by 43
  2. 2023Reproducing Whisper-Style Training Using an Open-Source Toolkit and Publicly Available Datacited by 42
  3. 2021An Exploration of Self-Supervised Pretrained Representations for End-to-End Speech Recognitioncited by 39
  4. 2022u-HuBERT: Unified Mixed-Modal Speech Pretraining And Zero-Shot Transfer to Unlabeled Modalitycited by 36
  5. 2025OWSM v4: Improving Open Whisper-Style Speech Models via Data Scaling and Cleaningcited by 27
  6. 2023Adaptation of Whisper models to child speech recognitioncited by 25
  7. 2021Injecting Text in Self-Supervised Speech Pretrainingcited by 24
  8. 2024Speech Translation with Speech Foundation Models and Large Language Models: What is There and What is Missing?cited by 23
  9. 2023Miipher: A Robust Speech Restoration Model Integrating Self-Supervised Speech and Text Representationscited by 22
  10. 2022ESB: A Benchmark For Multi-Domain End-to-End Speech Recognitioncited by 18
  11. 2022Accelerating RNN-T Training and Inference Using CTC guidancecited by 17
  12. 2021Scaling ASR Improves Zero and Few Shot Learningcited by 15
  13. 2024Measuring the Accuracy of Automatic Speech Recognition Solutionscited by 15
  14. 2022A Survey of Intent Classification and Slot-Filling Datasets for Task-Oriented Dialogcited by 14
  15. 2023Efficient Domain Adaptation for Speech Foundation Modelscited by 13
  16. 2023Multi-stage Large Language Model Correction for Speech Recognitioncited by 13
  17. 2021Spell my name: keyword boosted speech recognitioncited by 10
  18. 2022Boosting Cross-Domain Speech Recognition with Self-Supervisioncited by 10
  19. 2021CrowdSpeech and VoxDIY: Benchmark Datasets for Crowdsourced Audio Transcriptioncited by 9
  20. 2023UniverSLU: Universal Spoken Language Understanding for Diverse Tasks with Natural Language Instructionscited by 9
  21. 2025Loquacious Set: 25,000 Hours of Transcribed and Diverse English Speech Recognition Data for Research and Commercial Usecited by 9
  22. 2024Codec-ASR: Training Performant Automatic Speech Recognition Systems with Discrete Speech Representationscited by 8
  23. 2022Sample-Efficient Unsupervised Domain Adaptation of Speech Recognition Systems A case study for Modern Greekcited by 7
  24. 2023Modular Domain Adaptation for Conformer-Based Streaming ASRcited by 7
  25. 2023AfriNames: Most ASR models "butcher" African Namescited by 7
  26. 2024VoiceShop: A Unified Speech-to-Speech Framework for Identity-Preserving Zero-Shot Voice Editingcited by 7
  27. 2023Understanding Shared Speech-Text Representationscited by 6
  28. 2023Automatic Disfluency Detection from Untranscribed Speechcited by 6
  29. 2021A deep convolutional neural network that is invariant to time rescalingcited by 5
  30. 20223M: Multi-loss, Multi-path and Multi-level Neural Networks for speech recognitioncited by 5
  31. 2023AfroDigits: A Community-Driven Spoken Digit Dataset for African Languagescited by 4
  32. 2023Hyper-parameter Adaptation of Conformer ASR Systems for Elderly and Dysarthric Speech Recognitioncited by 4
  33. 2025Whisper-LM: Improving ASR Models with Language Models for Low-Resource Languagescited by 4
  34. 2025OLMoASR: Open Models and Data for Training Robust Speech Recognition Modelscited by 4
  35. 2025AgriGPT-Omni: A Unified Speech-Vision-Text Framework for Multilingual Agricultural Intelligencecited by 4
  36. 2022Ask2Mask: Guided Data Selection for Masked Speech Modelingcited by 3
  37. 2022Similarity and Content-based Phonetic Self Attention for Speech Recognitioncited by 3
  38. 2022Learning the joint distribution of two sequences using little or no paired datacited by 3

50 most cited of 83 All 83 (JSON)