mlsanityData to 7 Oct 2026

Noisy student-teacher training for robust keyword spotting

arXiv 2106.01604 ↗First posted 3 Jun 2021cs.LG stat.MLcited by 10

Not on the map. Why?

Abstract (arXiv)

We propose self-training with noisy student-teacher approach for streaming keyword spotting, that can utilize large-scale unlabeled data and aggressive data augmentation. The proposed method applies aggressive data augmentation (spectral augmentation) on the input of both student and teacher and utilize unlabeled data at scale, which significantly boosts the accuracy of student against challenging conditions. Such aggressive augmentation usually degrades model performance when used with supervised training with hard-labeled data. Experiments show that aggressive spec augmentation on baseline supervised training method degrades accuracy, while the proposed self-training with noisy student-teacher training improves accuracy of some difficult-conditioned test sets by as much as 60%.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 17 of its 25 references that are in the map. First posted on arXiv 3 Jun 2021.

Builds on

17 of 25 references.

References · 17

  1. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  3. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
  4. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,304
  5. 2019S4L: Self-Supervised Semi-Supervised Learningcited by 335
  6. 2018Streaming End-to-end Speech Recognition For Mobile Devicescited by 230
  7. 2019Revisiting Self-Training for Neural Sequence Generationcited by 154
  8. 2020Improved Noisy Student Training for Automatic Speech Recognitioncited by 139
  9. 2019Putting An End to End-to-End: Gradient-Isolated Learning of Representationscited by 85
  10. 2019Learning to Self-Train for Semi-Supervised Few-Shot Classificationcited by 83
  11. 2019Automatic adaptation of object detectors to new domains using self-trainingcited by 66
  12. 2019SpecAugment on Large Scale Datasetscited by 60
5 more
  1. 2018End-to-End Streaming Keyword Spottingcited by 25
  2. 2017Streaming Small-Footprint Keyword Spotting using Sequence-to-Sequence Modelscited by 19
  3. 2017A Cascade Architecture for Keyword Spotting on Mobile Devicescited by 15
  4. 2020Learning To Detect Keyword Parts And Whole By Smoothed Max Poolingcited by 10
  5. 2019Semi-supervised and Population Based Training for Voice Commands Recognitioncited by 1

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 3; 2023: 3; 2024: 2; 2025: 1.

10 papers.

  1. 2021Deep Spoken Keyword Spotting: An Overviewcited by 43
  2. 2022Production federated keyword spotting via distillation, filtering, and joint federated-centralized trainingcited by 9
  3. 2024Self-Learning for Personalized Keyword Spotting on Ultra-Low-Power Audio Sensorscited by 7
  4. 2022Low-resource Low-footprint Wake-word Detection using Knowledge Distillationcited by 3
  5. 2023Locale Encoding For Scalable Multilingual Keyword Spotting Modelscited by 3
  6. 2022Speech Augmentation Based Unsupervised Learning for Keyword Spottingcited by 1
  7. 2023Spot keywords from very noisy and mixed speechcited by 1
  8. 2023Personalizing Keyword Spotting with Speaker Informationcited by 1
  9. 2025Adaptive Noise Resilient Keyword Spotting Using One-Shot Learningcited by 1
  10. 2024Contrastive Augmentation: An Unsupervised Learning Approach for Keyword Spotting in Speech Technologycited by 0