# Personalized Keyphrase Detection using Speaker and Environment Information

arXiv: https://arxiv.org/abs/2104.13970 · HTML: https://mlsanity.com/paper/2104.13970/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 40 references that are in the map.
- First posted on arXiv 28 Apr 2021.

## Builds on (most cited first, 22 of 22)

- [VoxCeleb2: Deep Speaker Recognition](https://mlsanity.com/paper/1806.05622/) (2018)
- [Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions](https://mlsanity.com/paper/1712.05884/) (2017)
- [VoxCeleb: a large-scale speaker identification dataset](https://mlsanity.com/paper/1706.08612/) (2017)
- [To prune, or not to prune: exploring the efficacy of pruning for model compression](https://mlsanity.com/paper/1710.01878/) (2017)
- [LSTM: A Search Space Odyssey](https://mlsanity.com/paper/1503.04069/) (2015)
- [Generalized End-to-End Loss for Speaker Verification](https://mlsanity.com/paper/1710.10467/) (2017)
- [Streaming End-to-end Speech Recognition For Mobile Devices](https://mlsanity.com/paper/1811.06621/) (2018)
- [Deep clustering: Discriminative embeddings for segmentation and separation](https://mlsanity.com/paper/1508.04306/) (2015)
- [A Streaming On-Device End-to-End Model Surpassing Server-Side Conventional Model Quality and Latency](https://mlsanity.com/paper/2003.12710/) (2020)
- [Multi-talker Speech Separation with Utterance-level Permutation Invariant Training of Deep Recurrent Neural Networks](https://mlsanity.com/paper/1703.06284/) (2017)
- [Recognizing long-form speech using streaming end-to-end models](https://mlsanity.com/paper/1910.11455/) (2019)
- [VoiceFilter: Targeted Voice Separation by Speaker-Conditioned Spectrogram Masking](https://mlsanity.com/paper/1810.04826/) (2018)
- [CN-CELEB: a challenging Chinese speaker recognition dataset](https://mlsanity.com/paper/1911.01799/) (2019)
- [Voice Separation with an Unknown Number of Multiple Speakers](https://mlsanity.com/paper/2003.01531/) (2020)
- [Optimizing Speech Recognition For The Edge](https://mlsanity.com/paper/1909.12408/) (2019)
- [VoiceFilter-Lite: Streaming Targeted Voice Separation for On-Device Speech Recognition](https://mlsanity.com/paper/2009.04323/) (2020)
- [End-to-End Streaming Keyword Spotting](https://mlsanity.com/paper/1812.02802/) (2018)
- [Streaming Small-Footprint Keyword Spotting using Sequence-to-Sequence Models](https://mlsanity.com/paper/1710.09617/) (2017)
- [Attention-based End-to-End Models for Small-Footprint Keyword Spotting](https://mlsanity.com/paper/1803.10916/) (2018)
- [Dr-Vectors: Decision Residual Networks and an Improved Loss for Speaker Recognition](https://mlsanity.com/paper/2104.01989/) (2021)
- [Version Control of Speaker Recognition Systems](https://mlsanity.com/paper/2007.12069/) (2020)
- [Deep Karaoke: Extracting Vocals from Musical Mixtures Using a Convolutional Deep Neural Network](https://mlsanity.com/paper/1504.04658/) (2015)

## Built on by (most cited first, 8 of 8)

- [Deep Spoken Keyword Spotting: An Overview](https://mlsanity.com/paper/2111.10592/) (2021)
- [Turn-to-Diarize: Online Speaker Diarization Constrained by Transformer Transducer Speaker Turn Detection](https://mlsanity.com/paper/2109.11641/) (2021)
- [E3 TTS: Easy End-to-End Diffusion-based Text to Speech](https://mlsanity.com/paper/2311.00945/) (2023)
- [Multi-user VoiceFilter-Lite via Attentive Speaker Embedding](https://mlsanity.com/paper/2107.01201/) (2021)
- [Closing the Gap between Single-User and Multi-User VoiceFilter-Lite](https://mlsanity.com/paper/2202.12169/) (2022)
- [Improving Voice Trigger Detection with Metric Learning](https://mlsanity.com/paper/2204.02455/) (2022)
- [Keyword Spotting with Hyper-Matched Filters for Small Footprint Devices](https://mlsanity.com/paper/2508.04857/) (2025)
- [Training Intelligent Voice Assistant Wakeup with Controllable Synthetic Conversations](https://mlsanity.com/paper/2609.27037/) (2026)

## Abstract (arXiv)

In this paper, we introduce a streaming keyphrase detection system that can be easily customized to accurately detect any phrase composed of words from a large vocabulary. The system is implemented with an end-to-end trained automatic speech recognition (ASR) model and a text-independent speaker verification model. To address the challenge of detecting these keyphrases under various noisy conditions, a speaker separation model is added to the feature frontend of the speaker verification model, and an adaptive noise cancellation (ANC) algorithm is included to exploit cross-microphone noise coherence. Our experiments show that the text-independent speaker verification model largely reduces the false triggering rate of the keyphrase detection, while the speaker separation model and adaptive noise cancellation largely reduce false rejections.