# Teaching keyword spotters to spot new keywords with limited examples

arXiv: https://arxiv.org/abs/2106.02443 · HTML: https://mlsanity.com/paper/2106.02443/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 34 references that are in the map.
- First posted on arXiv 4 Jun 2021.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Common Voice: A Massively-Multilingual Speech Corpus](https://mlsanity.com/paper/1912.06670/) (2019)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition](https://mlsanity.com/paper/1804.03209/) (2018)
- [Contrastive Learning of General-Purpose Audio Representations](https://mlsanity.com/paper/2010.10915/) (2020)
- [Towards Learning a Universal Non-Semantic Representation of Speech](https://mlsanity.com/paper/2002.12764/) (2020)
- [Streaming keyword spotting on mobile devices](https://mlsanity.com/paper/2005.06720/) (2020)
- [MatchboxNet: 1D Time-Channel Separable Convolutional Neural Network Architecture for Speech Commands Recognition](https://mlsanity.com/paper/2004.08531/) (2020)
- [An Investigation Into On-device Personalization of End-to-end Automatic Speech Recognition Models](https://mlsanity.com/paper/1909.06678/) (2019)
- [Personalized Speech recognition on mobile devices](https://mlsanity.com/paper/1603.03185/) (2016)
- [Efficient keyword spotting using dilated convolutions and gating](https://mlsanity.com/paper/1811.07684/) (2018)
- [Training Keyword Spotters with Limited and Synthesized Speech Data](https://mlsanity.com/paper/2002.01322/) (2020)
- [Query-by-example on-device keyword spotting](https://mlsanity.com/paper/1910.05171/) (2019)
- [Few-Shot Keyword Spotting With Prototypical Networks](https://mlsanity.com/paper/2007.14463/) (2020)
- [DONUT: CTC-based Query-by-Example Keyword Spotting](https://mlsanity.com/paper/1811.10736/) (2018)
- [Towards Data-efficient Modeling for Wake Word Spotting](https://mlsanity.com/paper/2010.06659/) (2020)
- [Predicting detection filters for small footprint open-vocabulary keyword spotting](https://mlsanity.com/paper/1912.07575/) (2019)
- [Small-Footprint Open-Vocabulary Keyword Spotting with Quantized LSTM Networks](https://mlsanity.com/paper/2002.10851/) (2020)
- [Howl: A Deployed, Open-Source Wake Word Detection System](https://mlsanity.com/paper/2008.09606/) (2020)

## Built on by (most cited first, 9 of 9)

- [Progressive Continual Learning for Spoken Keyword Spotting](https://mlsanity.com/paper/2201.12546/) (2022)
- [Rainbow Keywords: Efficient Incremental Learning for Online Spoken Keyword Spotting](https://mlsanity.com/paper/2203.16361/) (2022)
- [Self-Learning for Personalized Keyword Spotting on Ultra-Low-Power Audio Sensors](https://mlsanity.com/paper/2408.12481/) (2024)
- [On the Efficiency of Integrating Self-supervised Learning and Meta-learning for User-defined Few-shot Keyword Spotting](https://mlsanity.com/paper/2204.00352/) (2022)
- [AnalyticKWS: Towards Exemplar-Free Analytic Class Incremental Learning for Small-footprint Keyword Spotting](https://mlsanity.com/paper/2505.11817/) (2025)
- [Locale Encoding For Scalable Multilingual Keyword Spotting Models](https://mlsanity.com/paper/2302.12961/) (2023)
- [Text-Aware Adapter for Few-Shot Keyword Spotting](https://mlsanity.com/paper/2412.18142/) (2024)
- [Towards noise robust trigger-word detection with contrastive learning pre-task for fast on-boarding of new trigger-words](https://mlsanity.com/paper/2111.03971/) (2021)
- [Enhancing Synthetic Training Data for Speech Commands: From ASR-Based Filtering to Domain Adaptation in SSL Latent Space](https://mlsanity.com/paper/2409.12745/) (2024)

## Abstract (arXiv)

Learning to recognize new keywords with just a few examples is essential for personalizing keyword spotting (KWS) models to a user's choice of keywords. However, modern KWS models are typically trained on large datasets and restricted to a small vocabulary of keywords, limiting their transferability to a broad range of unseen keywords. Towards easily customizable KWS models, we present KeySEM (Keyword Speech EMbedding), a speech embedding model pre-trained on the task of recognizing a large number of keywords. Speech representations offered by KeySEM are highly effective for learning new keywords from a limited number of examples. Comparisons with a diverse range of related work across several datasets show that our method achieves consistently superior performance with fewer training examples. Although KeySEM was pre-trained only on English utterances, the performance gains also extend to datasets from four other languages indicating that KeySEM learns useful representations well aligned with the task of keyword spotting. Finally, we demonstrate KeySEM's ability to learn new keywords sequentially without requiring to re-train on previously learned keywords. Our experimental observations suggest that KeySEM is well suited to on-device environments where post-deployment learning and ease of customization are often desirable.