# AnalyticKWS: Towards Exemplar-Free Analytic Class Incremental Learning for Small-footprint Keyword Spotting

arXiv: https://arxiv.org/abs/2505.11817 · HTML: https://mlsanity.com/paper/2505.11817/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 37 references that are in the map.
- First posted on arXiv 17 May 2025.

## Builds on (most cited first, 23 of 23)

- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [Riemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence](https://mlsanity.com/paper/1801.10112/) (2018)
- [Large Scale Incremental Learning](https://mlsanity.com/paper/1905.13260/) (2019)
- [Online Structured Laplace Approximations For Overcoming Catastrophic Forgetting](https://mlsanity.com/paper/1805.07810/) (2018)
- [Hello Edge: Keyword Spotting on Microcontrollers](https://mlsanity.com/paper/1711.07128/) (2017)
- [A Comprehensive Study of Class Incremental Learning Algorithms for Visual Tasks](https://mlsanity.com/paper/2011.01844/) (2020)
- [FeCAM: Exploiting the Heterogeneity of Class Distributions in Exemplar-Free Continual Learning](https://mlsanity.com/paper/2309.14062/) (2023)
- [Deep Residual Learning for Small-Footprint Keyword Spotting](https://mlsanity.com/paper/1710.10361/) (2017)
- [Temporal Convolution for Real-time Keyword Spotting on Mobile Devices](https://mlsanity.com/paper/1904.03814/) (2019)
- [Broadcasted Residual Learning for Efficient Keyword Spotting](https://mlsanity.com/paper/2106.04140/) (2021)
- [Deep Spoken Keyword Spotting: An Overview](https://mlsanity.com/paper/2111.10592/) (2021)
- [Resurrecting Old Classes with New Data for Exemplar-Free Continual Learning](https://mlsanity.com/paper/2405.19074/) (2024)
- [Few-Shot Keyword Spotting in Any Language](https://mlsanity.com/paper/2104.01454/) (2021)
- [Towards Exemplar-Free Continual Learning in Vision Transformers: an Account of Attention, Functional and Weight Regularization](https://mlsanity.com/paper/2203.13167/) (2022)
- [Online Continual Learning of End-to-End Speech Recognition Models](https://mlsanity.com/paper/2207.05071/) (2022)
- [Teaching keyword spotters to spot new keywords with limited examples](https://mlsanity.com/paper/2106.02443/) (2021)
- [Progressive Continual Learning for Spoken Keyword Spotting](https://mlsanity.com/paper/2201.12546/) (2022)
- [Rainbow Keywords: Efficient Incremental Learning for Online Spoken Keyword Spotting](https://mlsanity.com/paper/2203.16361/) (2022)
- [Dark Experience for Incremental Keyword Spotting](https://mlsanity.com/paper/2409.08153/) (2024)
- [ED-sKWS: Early-Decision Spiking Neural Networks for Rapid,and Energy-Efficient Keyword Spotting](https://mlsanity.com/paper/2406.12726/) (2024)

## Built on by (most cited first, 4 of 4)

- [Adapting Where It Matters: Depth-Aware Adaptation for Efficient Multilingual Speech Recognition in Low-Resource Languages](https://mlsanity.com/paper/2602.01008/) (2026)
- [Keyword Mamba: Spoken Keyword Spotting with State Space Models](https://mlsanity.com/paper/2508.07363/) (2025)
- [Continual Adaptation for Pacific Indigenous Speech Recognition](https://mlsanity.com/paper/2603.06310/) (2026)
- [SPARCL: Spectral Partitioned Analytic Continual Learning](https://mlsanity.com/paper/2608.21307/) (2026)

## Abstract (arXiv)

Keyword spotting (KWS) offers a vital mechanism to identify spoken commands in voice-enabled systems, where user demands often shift, requiring models to learn new keywords continually over time. However, a major problem is catastrophic forgetting, where models lose their ability to recognize earlier keywords. Although several continual learning methods have proven their usefulness for reducing forgetting, most existing approaches depend on storing and revisiting old data to combat catastrophic forgetting. Though effective, these methods face two practical challenges: 1) privacy risks from keeping user data and 2) large memory and time consumption that limit deployment on small devices. To address these issues, we propose an exemplar-free Analytic Continual Learning (AnalyticKWS) method that updates model parameters without revisiting earlier data. Inspired by efficient learning principles, AnalyticKWS computes a closed-form analytical solution for model updates and requires only a single epoch of adaptation for incoming keywords. AnalyticKWS demands fewer computational resources by avoiding gradient-based updates and does not store old data. By eliminating the need for back-propagation during incremental learning, the model remains lightweight and efficient. As a result, AnalyticKWS meets the challenges mentioned earlier and suits resource-limited settings well. Extensive experiments on various datasets and settings show that AnalyticKWS consistently outperforms existing continual learning methods.