# Low-resource Low-footprint Wake-word Detection using Knowledge Distillation

arXiv: https://arxiv.org/abs/2207.03331 · HTML: https://mlsanity.com/paper/2207.03331/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Speech and audio distillation, one of 12 directions in Knowledge distillation.
- Builds on 10 of its 22 references that are in the map.
- First posted on arXiv 6 Jul 2022.

## Placed in

- Knowledge distillation / [Speech and audio distillation](https://mlsanity.com/distill/speech-and-audio-distillation/)

## Builds on (most cited first, 10 of 10)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [MUSAN: A Music, Speech, and Noise Corpus](https://mlsanity.com/paper/1510.08484/) (2015)
- [Efficient keyword spotting using dilated convolutions and gating](https://mlsanity.com/paper/1811.07684/) (2018)
- [Knowledge Distillation for Small-footprint Highway Networks](https://mlsanity.com/paper/1608.00892/) (2016)
- [Towards Data-efficient Modeling for Wake Word Spotting](https://mlsanity.com/paper/2010.06659/) (2020)
- [Noisy student-teacher training for robust keyword spotting](https://mlsanity.com/paper/2106.01604/) (2021)
- [Wake Word Detection with Alignment-Free Lattice-Free MMI](https://mlsanity.com/paper/2005.08347/) (2020)
- [Knowledge Transfer for Efficient On-device False Trigger Mitigation](https://mlsanity.com/paper/2010.10591/) (2020)

## Built on by (most cited first, 3 of 3)

- [SELMA: A Speech-Enabled Language Model for Virtual Assistant Interactions](https://mlsanity.com/paper/2501.19377/) (2025)
- [Multimodal Data and Resource Efficient Device-Directed Speech Detection with Large Foundation Models](https://mlsanity.com/paper/2312.03632/) (2023)
- [A Multimodal Approach to Device-Directed Speech Detection with Large Language Models](https://mlsanity.com/paper/2403.14438/) (2024)

## Abstract (arXiv)

As virtual assistants have become more diverse and specialized, so has the demand for application or brand-specific wake words. However, the wake-word-specific datasets typically used to train wake-word detectors are costly to create. In this paper, we explore two techniques to leverage acoustic modeling data for large-vocabulary speech recognition to improve a purpose-built wake-word detector: transfer learning and knowledge distillation. We also explore how these techniques interact with time-synchronous training targets to improve detection latency. Experiments are presented on the open-source "Hey Snips" dataset and a more challenging in-house far-field dataset. Using phone-synchronous targets and knowledge distillation from a large acoustic model, we are able to improve accuracy across dataset sizes for both datasets while reducing latency.