# Towards noise robust trigger-word detection with contrastive learning pre-task for fast on-boarding of new trigger-words

arXiv: https://arxiv.org/abs/2111.03971 · HTML: https://mlsanity.com/paper/2111.03971/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 25 references that are in the map.
- First posted on arXiv 6 Nov 2021.

## Builds on (most cited first, 17 of 17)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Self-Supervised Learning of Pretext-Invariant Representations](https://mlsanity.com/paper/1912.01991/) (2019)
- [Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition](https://mlsanity.com/paper/1804.03209/) (2018)
- [Transfer learning for music classification and regression tasks](https://mlsanity.com/paper/1703.09179/) (2017)
- [Mining on Manifolds: Metric Learning without Labels](https://mlsanity.com/paper/1803.11095/) (2018)
- [Self-supervised Text-independent Speaker Verification using Prototypical Momentum Contrastive Learning](https://mlsanity.com/paper/2012.07178/) (2020)
- [CLAR: Contrastive Learning of Auditory Representations](https://mlsanity.com/paper/2010.09542/) (2020)
- [Attention-based End-to-End Models for Small-Footprint Keyword Spotting](https://mlsanity.com/paper/1803.10916/) (2018)
- [Towards Data-efficient Modeling for Wake Word Spotting](https://mlsanity.com/paper/2010.06659/) (2020)
- [Wake Word Detection with Streaming Transformers](https://mlsanity.com/paper/2102.04488/) (2021)
- [Teaching keyword spotters to spot new keywords with limited examples](https://mlsanity.com/paper/2106.02443/) (2021)
- [Efficient keyword spotting using time delay neural networks](https://mlsanity.com/paper/1807.04353/) (2018)
- [Self-Supervised Learning from Contrastive Mixtures for Personalized Speech Enhancement](https://mlsanity.com/paper/2011.03426/) (2020)
- [SoundCLR: Contrastive Learning of Representations For Improved Environmental Sound Classification](https://mlsanity.com/paper/2103.01929/) (2021)

## Abstract (arXiv)

Trigger-word detection plays an important role as the entry point of user's communication with voice assistants. But supporting a particular word as a trigger-word involves huge amount of data collection, augmentation and labelling for that word. This makes supporting new trigger-words a tedious and time consuming process. To combat this, we explore the use of contrastive learning as a pre-training task that helps the detection model to generalize to different words and noise conditions. We explore supervised contrastive techniques and also propose a novel self-supervised training technique using chunked words from long sentence audios. We show that both supervised and the new self-supervised contrastive pre-training techniques have comparable results to a traditional classification pre-training on new trigger words with less data availability.