# Query-by-Example Keyword Spotting system using Multi-head Attention and Softtriple Loss

arXiv: https://arxiv.org/abs/2102.07061 · HTML: https://mlsanity.com/paper/2102.07061/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 20 references that are in the map.
- First posted on arXiv 14 Feb 2021.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [In Defense of the Triplet Loss for Person Re-Identification](https://mlsanity.com/paper/1703.07737/) (2017)
- [SoftTriple Loss: Deep Metric Learning Without Triplet Sampling](https://mlsanity.com/paper/1909.05235/) (2019)
- [Deep Residual Learning for Small-Footprint Keyword Spotting](https://mlsanity.com/paper/1710.10361/) (2017)
- [Discriminative Acoustic Word Embeddings: Recurrent Neural Network-Based Approaches](https://mlsanity.com/paper/1611.02550/) (2016)
- [Efficient keyword spotting using dilated convolutions and gating](https://mlsanity.com/paper/1811.07684/) (2018)
- [End-to-End ASR-free Keyword Search from Speech](https://mlsanity.com/paper/1701.04313/) (2017)
- [Streaming Small-Footprint Keyword Spotting using Sequence-to-Sequence Models](https://mlsanity.com/paper/1710.09617/) (2017)
- [Attention-Based Models for Text-Dependent Speaker Verification](https://mlsanity.com/paper/1710.10470/) (2017)
- [DONUT: CTC-based Query-by-Example Keyword Spotting](https://mlsanity.com/paper/1811.10736/) (2018)
- [Query-by-example Spoken Term Detection using Attention-based Multi-hop Networks](https://mlsanity.com/paper/1709.00354/) (2017)

## Built on by (most cited first, 25 of 25)

- [Deep Spoken Keyword Spotting: An Overview](https://mlsanity.com/paper/2111.10592/) (2021)
- [Few-Shot Keyword Spotting in Any Language](https://mlsanity.com/paper/2104.01454/) (2021)
- [Learning Audio-Text Agreement for Open-vocabulary Keyword Spotting](https://mlsanity.com/paper/2206.15400/) (2022)
- [Matching Latent Encoding for Audio-Text based Keyword Spotting](https://mlsanity.com/paper/2306.05245/) (2023)
- [Few-Shot Open-Set Learning for On-Device Customization of KeyWord Spotting Systems](https://mlsanity.com/paper/2306.02161/) (2023)
- [Open-vocabulary Keyword-spotting with Adaptive Instance Normalization](https://mlsanity.com/paper/2309.08561/) (2023)
- [Dummy Prototypical Networks for Few-Shot Open-Set Keyword Spotting](https://mlsanity.com/paper/2206.13691/) (2022)
- [Personalized Keyword Spotting through Multi-task Learning](https://mlsanity.com/paper/2206.13708/) (2022)
- [On the Efficiency of Integrating Self-supervised Learning and Meta-learning for User-defined Few-shot Keyword Spotting](https://mlsanity.com/paper/2204.00352/) (2022)
- [QbyE-MLPMixer: Query-by-Example Open-Vocabulary Keyword Spotting using MLPMixer](https://mlsanity.com/paper/2206.13231/) (2022)
- [MM-KWS: Multi-modal Prompts for Multilingual User-defined Keyword Spotting](https://mlsanity.com/paper/2406.07310/) (2024)
- [Relational Proxy Loss for Audio-Text based Keyword Spotting](https://mlsanity.com/paper/2406.05314/) (2024)
- [Text-Aware Adapter for Few-Shot Keyword Spotting](https://mlsanity.com/paper/2412.18142/) (2024)
- [Improving Voice Trigger Detection with Metric Learning](https://mlsanity.com/paper/2204.02455/) (2022)
- [End-to-End User-Defined Keyword Spotting using Shifted Delta Coefficients](https://mlsanity.com/paper/2405.14489/) (2024)
- [Adversarial Deep Metric Learning for Cross-Modal Audio-Text Alignment in Open-Vocabulary Keyword Spotting](https://mlsanity.com/paper/2505.16735/) (2025)
- [Continuously Reliable Detection of New-Normal Misinformation: Semantic Masking and Contrastive Smoothing in High-Density Latent Regions](https://mlsanity.com/paper/2301.07981/) (2023)
- [Improving Small Footprint Few-shot Keyword Spotting with Supervision on Auxiliary Data](https://mlsanity.com/paper/2309.00647/) (2023)
- [Query-by-Example Keyword Spotting Using Spectral-Temporal Graph Attentive Pooling and Multi-Task Learning](https://mlsanity.com/paper/2409.00099/) (2024)
- [SLiCK: Exploiting Subsequences for Length-Constrained Keyword Spotting](https://mlsanity.com/paper/2409.09067/) (2024)
- [Noise-Agnostic Multitask Whisper Training for Reducing False Alarm Errors in Call-for-Help Detection](https://mlsanity.com/paper/2501.11631/) (2025)
- [Keyword Spotting with Hyper-Matched Filters for Small Footprint Devices](https://mlsanity.com/paper/2508.04857/) (2025)
- [MATE: Matryoshka Audio-Text Embeddings for Open-Vocabulary Keyword Spotting](https://mlsanity.com/paper/2601.14012/) (2026)
- [PCOV-KWS: Multi-task Learning for Personalized Customizable Open Vocabulary Keyword Spotting](https://mlsanity.com/paper/2603.18023/) (2026)
- [ProKWS: Personalized Keyword Spotting via Collaborative Learning of Phonemes and Prosody](https://mlsanity.com/paper/2603.18024/) (2026)

## Abstract (arXiv)

This paper proposes a neural network architecture for tackling the query-by-example user-defined keyword spotting task. A multi-head attention module is added on top of a multi-layered GRU for effective feature extraction, and a normalized multi-head attention module is proposed for feature aggregation. We also adopt the softtriple loss - a combination of triplet loss and softmax loss - and showcase its effectiveness. We demonstrate the performance of our model on internal datasets with different languages and the public Hey-Snips dataset. We compare the performance of our model to a baseline system and conduct an ablation study to show the benefit of each component in our architecture. The proposed work shows solid performance while preserving simplicity.