mlsanityData to 7 Oct 2026

Rainbow Keywords: Efficient Incremental Learning for Online Spoken Keyword Spotting

arXiv 2203.16361 ↗First posted 30 Mar 2022cs.SD cs.CL eess.AScited by 8

Not on the map. Why?

Abstract (arXiv)

Catastrophic forgetting is a thorny challenge when updating keyword spotting (KWS) models after deployment. This problem will be more challenging if KWS models are further required for edge devices due to their limited memory. To alleviate such an issue, we propose a novel diversity-aware incremental learning method named Rainbow Keywords (RK). Specifically, the proposed RK approach introduces a diversity-aware sampler to select a diverse set from historical and incoming keywords by calculating classification uncertainty. As a result, the RK approach can incrementally learn new tasks without forgetting prior knowledge. Besides, the RK approach also proposes data augmentation and knowledge distillation loss function for efficient memory management on the edge device. Experimental results show that the proposed RK approach achieves 4.2% absolute improvement in terms of average accuracy over the best baseline on Google Speech Command dataset with less required memory. The scripts are available on GitHub.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 33 of its 51 references that are in the map. First posted on arXiv 30 Mar 2022.

Builds on

33 of 51 references.

References · 33

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  3. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  4. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  5. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  6. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
  7. 2017Gradient Episodic Memory for Continual Learningcited by 1,871
  8. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
  9. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,313
  10. 2018Speech Commands: A Dataset for Limited-Vocabulary Speech Recognitioncited by 741
  11. 2018Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigencecited by 734
  12. 2019Large Scale Incremental Learningcited by 727
21 more
  1. 2020Class-incremental learning: survey and performance evaluation on image classificationcited by 446
  2. 2018Meta-Transfer Learning for Few-Shot Learningcited by 306
  3. 2021Rainbow Memory: Continual Learning with a Memory of Diverse Samplescited by 273
  4. 2021Online Continual Learning in Image Classification: An Empirical Surveycited by 242
  5. 2018Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselinescited by 228
  6. 2017Hello Edge: Keyword Spotting on Microcontrollerscited by 108
  7. 2017Deep Residual Learning for Small-Footprint Keyword Spottingcited by 68
  8. 2021Keyword Transformer: A Self-Attention Model for Keyword Spottingcited by 59
  9. 2019Temporal Convolution for Real-time Keyword Spotting on Mobile Devicescited by 48
  10. 2021Broadcasted Residual Learning for Efficient Keyword Spottingcited by 47
  11. 2021Deep Spoken Keyword Spotting: An Overviewcited by 43
  12. 2021Few-Shot Keyword Spotting in Any Languagecited by 27
  13. 2020Training Keyword Spotters with Limited and Synthesized Speech Datacited by 20
  14. 2020MLModelCI: An Automatic Cloud Platform for Efficient MLaaScited by 16
  15. 2022ConvMixer: Feature Interactive Convolution with Curriculum Learning for Small Footprint and Noisy Far-field Keyword Spottingcited by 11
  16. 2021A Serverless Cloud-Fog Platform for DNN-Based Video Analytics with Incremental Learningcited by 9
  17. 2021Teaching keyword spotters to spot new keywords with limited examplescited by 9
  18. 2021ModelCI-e: Enabling Continual Learning in Deep Learning Serving Systemscited by 9
  19. 2022Progressive Continual Learning for Spoken Keyword Spottingcited by 8
  20. 2021Audiomer: A Convolutional Transformer For Keyword Spottingcited by 4
  21. 2021ModelPS: An Interactive and Collaborative Platform for Editing Pre-trained Models at Scalecited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 2; 2024: 2; 2025: 1; 2026: 2.

8 papers.

  1. 2023Online Continual Learning in Keyword Spotting for Low-Resource Devices via Pooling High-Order Temporal Statisticscited by 9
  2. 2022Continual Learning For On-Device Environmental Sound Classificationcited by 4
  3. 2025AnalyticKWS: Towards Exemplar-Free Analytic Class Incremental Learning for Small-footprint Keyword Spottingcited by 4
  4. 2026Adapting Where It Matters: Depth-Aware Adaptation for Efficient Multilingual Speech Recognition in Low-Resource Languagescited by 1
  5. 2023Conditional Online Learning for Keyword Spottingcited by 0
  6. 2024Evaluating and Improving Continual Learning in Spoken Language Understandingcited by 0
  7. 2024Advancing Airport Tower Command Recognition: Integrating Squeeze-and-Excitation and Broadcasted Residual Learningcited by 0
  8. 2026Continual Adaptation for Pacific Indigenous Speech Recognitioncited by 0