# Dark Experience for Incremental Keyword Spotting

arXiv: https://arxiv.org/abs/2409.08153 · HTML: https://mlsanity.com/paper/2409.08153/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Sep 2024.

## Built on by (most cited first, 4 of 4)

- [AnalyticKWS: Towards Exemplar-Free Analytic Class Incremental Learning for Small-footprint Keyword Spotting](https://mlsanity.com/paper/2505.11817/) (2025)
- [Adapting Where It Matters: Depth-Aware Adaptation for Efficient Multilingual Speech Recognition in Low-Resource Languages](https://mlsanity.com/paper/2602.01008/) (2026)
- [Keyword Mamba: Spoken Keyword Spotting with State Space Models](https://mlsanity.com/paper/2508.07363/) (2025)
- [Continual Adaptation for Pacific Indigenous Speech Recognition](https://mlsanity.com/paper/2603.06310/) (2026)

## Abstract (arXiv)

Spoken keyword spotting (KWS) is crucial for identifying keywords within audio inputs and is widely used in applications like Apple Siri and Google Home, particularly on edge devices. Current deep learning-based KWS systems, which are typically trained on a limited set of keywords, can suffer from performance degradation when encountering new domains, a challenge often addressed through few-shot fine-tuning. However, this adaptation frequently leads to catastrophic forgetting, where the model's performance on original data deteriorates. Progressive continual learning (CL) strategies have been proposed to overcome this, but they face limitations such as the need for task-ID information and increased storage, making them less practical for lightweight devices. To address these challenges, we introduce Dark Experience for Keyword Spotting (DE-KWS), a novel CL approach that leverages dark knowledge to distill past experiences throughout the training process. DE-KWS combines rehearsal and distillation, using both ground truth labels and logits stored in a memory buffer to maintain model performance across tasks. Evaluations on the Google Speech Command dataset show that DE-KWS outperforms existing CL baselines in average accuracy without increasing model size, offering an effective solution for resource-constrained edge devices. The scripts are available on GitHub for the future research.