# Few-Shot Open-Set Learning for On-Device Customization of KeyWord Spotting Systems

arXiv: https://arxiv.org/abs/2306.02161 · HTML: https://mlsanity.com/paper/2306.02161/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 26 references that are in the map.
- First posted on arXiv 3 Jun 2023.

## Builds on (most cited first, 19 of 19)

- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition](https://mlsanity.com/paper/1804.03209/) (2018)
- [Towards Open Set Deep Networks](https://mlsanity.com/paper/1511.06233/) (2015)
- [In defence of metric learning for speaker recognition](https://mlsanity.com/paper/2003.11982/) (2020)
- [Hello Edge: Keyword Spotting on Microcontrollers](https://mlsanity.com/paper/1711.07128/) (2017)
- [A Wholistic View of Continual Learning with Deep Neural Networks: Forgotten Lessons and the Bridge to Active and Open World Learning](https://mlsanity.com/paper/2009.01797/) (2020)
- [Few-Shot Open-Set Recognition using Meta-Learning](https://mlsanity.com/paper/2005.13713/) (2020)
- [Online Continual Learning for Embedded Devices](https://mlsanity.com/paper/2203.10681/) (2022)
- [Streaming keyword spotting on mobile devices](https://mlsanity.com/paper/2005.06720/) (2020)
- [Deep Spoken Keyword Spotting: An Overview](https://mlsanity.com/paper/2111.10592/) (2021)
- [Few-Shot Keyword Spotting in Any Language](https://mlsanity.com/paper/2104.01454/) (2021)
- [Query-by-Example Keyword Spotting system using Multi-head Attention and Softtriple Loss](https://mlsanity.com/paper/2102.07061/) (2021)
- [Learning Efficient Representations for Keyword Spotting with Triplet Loss](https://mlsanity.com/paper/2101.04792/) (2021)
- [Few-Shot Keyword Spotting With Prototypical Networks](https://mlsanity.com/paper/2007.14463/) (2020)
- [Metric Learning for Keyword Spotting](https://mlsanity.com/paper/2005.08776/) (2020)
- [Dummy Prototypical Networks for Few-Shot Open-Set Keyword Spotting](https://mlsanity.com/paper/2206.13691/) (2022)
- [Personalized Keyword Spotting through Multi-task Learning](https://mlsanity.com/paper/2206.13708/) (2022)
- [Metric Learning for User-defined Keyword Spotting](https://mlsanity.com/paper/2211.00439/) (2022)
- [QbyE-MLPMixer: Query-by-Example Open-Vocabulary Keyword Spotting using MLPMixer](https://mlsanity.com/paper/2206.13231/) (2022)

## Built on by (most cited first, 9 of 9)

- [Self-Learning for Personalized Keyword Spotting on Ultra-Low-Power Audio Sensors](https://mlsanity.com/paper/2408.12481/) (2024)
- [TinySV: Speaker Verification in TinyML with On-device Learning](https://mlsanity.com/paper/2406.01655/) (2024)
- [Few-Shot Open-Set Audio Classification Using Attention Information-Fused Prototypes](https://mlsanity.com/paper/2607.01297/) (2026)
- [Enhancing Few-shot Keyword Spotting Performance through Pre-Trained Self-supervised Speech Models](https://mlsanity.com/paper/2506.17686/) (2025)
- [EdgeSpot: Efficient and High-Performance Few-Shot Model for Keyword Spotting](https://mlsanity.com/paper/2601.16316/) (2026)
- [OnDA: On-device Channel Pruning for Efficient Personalized Keyword Spotting](https://mlsanity.com/paper/2603.02247/) (2026)
- [Scaling few-shot spoken word classification with generative meta-continual learning](https://mlsanity.com/paper/2605.13075/) (2026)
- [What changes after deployment? A survey on On-device Learning in TinyML](https://mlsanity.com/paper/2605.31226/) (2026)
- [Few-Shot Open-Set Audio Classification via Transductive Prototype Refinement and Class Logit Enhancement](https://mlsanity.com/paper/2607.26607/) (2026)

## Abstract (arXiv)

A personalized KeyWord Spotting (KWS) pipeline typically requires the training of a Deep Learning model on a large set of user-defined speech utterances, preventing fast customization directly applied on-device. To fill this gap, this paper investigates few-shot learning methods for open-set KWS classification by combining a deep feature encoder with a prototype-based classifier. With user-defined keywords from 10 classes of the Google Speech Command dataset, our study reports an accuracy of up to 76% in a 10-shot scenario while the false acceptance rate of unknown data is kept to 5%. In the analyzed settings, the usage of the triplet loss to train an encoder with normalized output features performs better than the prototypical networks jointly trained with a generator of dummy unknown-class prototypes. This design is also more effective than encoders trained on a classification problem and features fewer parameters than other iso-accuracy approaches.