mlsanityData to 14 Oct 2026

SpecAugment on Large Scale Datasets

arXiv 1912.05533 ↗First posted 11 Dec 2019eess.AS cs.CL cs.LGcited by 60

Not on the map. Why?

Abstract (arXiv)

Recently, SpecAugment, an augmentation scheme for automatic speech recognition that acts directly on the spectrogram of input utterances, has shown to be highly effective in enhancing the performance of end-to-end networks on public datasets. In this paper, we demonstrate its effectiveness on tasks with large scale datasets by investigating its application to the Google Multidomain Dataset (Narayanan et al., 2018). We achieve improvement across all test domains by mixing raw training data augmented with SpecAugment and noise-perturbed training data when training the acoustic model. We also introduce a modification of SpecAugment that adapts the time mask size and/or multiplicity depending on the length of the utterance, which can potentially benefit large scale tasks. By using adaptive masking, we are able to further improve the performance of the Listen, Attend and Spell model on LibriSpeech to 2.2% WER on test-clean and 5.2% WER on test-other.

Key facts. Cited by 60 papers in the map (data through 14 Oct 2026). First posted on arXiv 11 Dec 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 18; 2022: 21; 2023: 11; 2024: 6; 2025: 3; 2026: 1.

60 papers.

Sort:

By area:

  1. 2021W2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Trainingcited by 347
  2. 2023End-to-End Speech Recognition: A Surveycited by 129
  3. 2021BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognitioncited by 88
  4. 2021SpeechStew: Simply Mix All Available Speech Recognition Data to Train One Large Neural Networkcited by 83
  5. 2021A Practical Survey on Faster and Lighter Transformerscited by 73
  6. 2021Efficient conformer: Progressive downsampling and grouped attention for automatic speech recognitioncited by 42
  7. 2022Improving End-to-End Contextual Speech Recognition with Fine-Grained Contextual Knowledge Selectioncited by 39
  8. 2022Streaming Multi-Talker ASR with Token-Level Serialized Output Trainingcited by 37
  9. 2021Tied & Reduced RNN-T Decodercited by 36
  10. 2021SynthASR: Unlocking Synthetic Data for Speech Recognitioncited by 34
  11. 2023Audio-Visual Efficient Conformer for Robust Speech Recognitioncited by 22
  12. 2022SPIRAL: Self-supervised Perturbation-Invariant Representation Learning for Speech Pre-Trainingcited by 19
38 more
  1. 2022Foundation Transformerscited by 19
  2. 2021Pushing the Limits of Non-Autoregressive Speech Recognitioncited by 18
  3. 2022Interspace Pruning: Using Adaptive Filter Representations to Improve Training of Sparse CNNscited by 16
  4. 2022Efficient Training of Neural Transducer for Speech Recognitioncited by 14
  5. 2021Deliberation of Streaming RNN-Transducer by Non-autoregressive Decodingcited by 13
  6. 2023Phoneme-aware Encoding for Prefix-tree-based Contextual ASRcited by 13
  7. 2022Data Augmentation for Electrocardiogramscited by 12
  8. 2022Improving Rare Word Recognition with LM-aware MWER Trainingcited by 12
  9. 2022Dimensionality Reduced Training by Pruning and Freezing Parts of a Deep Neural Network, a Surveycited by 12
  10. 2023One model to rule them all ? Towards End-to-End Joint Speaker Diarization and Speech Recognitioncited by 12
  11. 2022Improving the fusion of acoustic and text representations in RNN-Tcited by 11
  12. 2023Self-Supervised Learning for Few-Shot Bird Sound Classificationcited by 11
  13. 2024HINT: High-quality INPainting Transformer with Mask-Aware Encoding and Enhanced Attentioncited by 11
  14. 2021Noisy student-teacher training for robust keyword spottingcited by 10
  15. 2021Multi-turn RNN-T for streaming recognition of multi-party speechcited by 10
  16. 2022A Language Agnostic Multilingual Streaming On-Device ASR Systemcited by 9
  17. 2022Simple Pooling Front-ends For Efficient Audio Classificationcited by 9
  18. 2021Multi-mode Transformer Transducer with Stochastic Future Contextcited by 7
  19. 2022Extracting Targeted Training Data from ASR Models, and How to Mitigate Itcited by 7
  20. 2024Multilingual Audio-Visual Speech Recognition with Hybrid CTC/RNN-T Fast Conformercited by 7
  21. 2022Improving Mandarin Speech Recogntion with Block-augmented Transformercited by 6
  22. 2021Advancing CTC-CRF Based End-to-End Speech Recognition with Wordpieces and Conformerscited by 4
  23. 2024Mixture of Mixups for Multi-label Classification of Rare Anuran Soundscited by 4
  24. 2021PM-MMUT: Boosted Phone-Mask Data Augmentation using Multi-Modeling Unit Training for Phonetic-Reduction-Robust E2E Speech Recognitioncited by 3
  25. 2022Separator-Transducer-Segmenter: Streaming Recognition and Segmentation of Multi-party Speechcited by 3
  26. 2025Speaker-Distinguishable CTC: Learning Speaker Distinction Using CTC for Multi-Talker Speech Recognitioncited by 3
  27. 2021Noisy Training Improves E2E ASR for the Edgecited by 2
  28. 2022G-Augment: Searching for the Meta-Structure of Data Augmentation Policies for ASRcited by 2
  29. 2023KinSPEAK: Improving speech recognition for Kinyarwanda via semi-supervised learning methodscited by 2
  30. 2023Harnessing the Zero-Shot Power of Instruction-Tuned Large Language Model in End-to-End Speech Recognitioncited by 2
  31. 2024Tailored Design of Audio-Visual Speech Recognition Models using Branchformerscited by 2
  32. 2024Improving noisy student training for low-resource languages in End-to-End ASR using CycleGAN and inter-domain lossescited by 2
  33. 2025Recent Trends in Distant Conversational Speech Recognition: A Review of CHiME-7 and 8 DASR Challengescited by 2
  34. 2021A Dual-Decoder Conformer for Multilingual Speech Recognitioncited by 1
  35. 2022Speech Augmentation Based Unsupervised Learning for Keyword Spottingcited by 1
  36. 2022Contextual-Utterance Training for Automatic Speech Recognitioncited by 1
  37. 2022Visual Transformers for Primates Classification and Covid Detectioncited by 1
  38. 2023Efficiency-oriented approaches for self-supervised speech representation learningcited by 1

50 most cited of 60 All 60 (JSON)