# Open-vocabulary Keyword-spotting with Adaptive Instance Normalization

arXiv: https://arxiv.org/abs/2309.08561 · HTML: https://mlsanity.com/paper/2309.08561/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 20 references that are in the map.
- First posted on arXiv 13 Sep 2023.

## Builds on (most cited first, 16 of 16)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [FLEURS: Few-shot Learning Evaluation of Universal Representations of Speech](https://mlsanity.com/paper/2205.12446/) (2022)
- [VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation](https://mlsanity.com/paper/2101.00390/) (2021)
- [Keyword Transformer: A Self-Attention Model for Keyword Spotting](https://mlsanity.com/paper/2104.00769/) (2021)
- [Deep Spoken Keyword Spotting: An Overview](https://mlsanity.com/paper/2111.10592/) (2021)
- [MatchboxNet: 1D Time-Channel Separable Convolutional Neural Network Architecture for Speech Commands Recognition](https://mlsanity.com/paper/2004.08531/) (2020)
- [Query-by-Example Keyword Spotting system using Multi-head Attention and Softtriple Loss](https://mlsanity.com/paper/2102.07061/) (2021)
- [Learning Audio-Text Agreement for Open-vocabulary Keyword Spotting](https://mlsanity.com/paper/2206.15400/) (2022)
- [DONUT: CTC-based Query-by-Example Keyword Spotting](https://mlsanity.com/paper/1811.10736/) (2018)
- [Matching Latent Encoding for Audio-Text based Keyword Spotting](https://mlsanity.com/paper/2306.05245/) (2023)
- [Flexible Keyword Spotting based on Homogeneous Audio-Text Embedding](https://mlsanity.com/paper/2308.06472/) (2023)
- [Metric Learning for User-defined Keyword Spotting](https://mlsanity.com/paper/2211.00439/) (2022)
- [CNN-based Spoken Term Detection and Localization without Dynamic Programming](https://mlsanity.com/paper/2103.05468/) (2021)
- [Fully Unsupervised Training of Few-shot Keyword Spotting](https://mlsanity.com/paper/2210.02732/) (2022)
- [Multilingual Query-by-Example Keyword Spotting with Metric Learning and Phoneme-to-Embedding Mapping](https://mlsanity.com/paper/2304.09585/) (2023)

## Built on by (most cited first, 8 of 8)

- [A Multitask Training Approach to Enhance Whisper with Contextual Biasing and Open-Vocabulary Keyword Spotting](https://mlsanity.com/paper/2309.09552/) (2023)
- [Keyword-Guided Adaptation of Automatic Speech Recognition](https://mlsanity.com/paper/2406.02649/) (2024)
- [MM-KWS: Multi-modal Prompts for Multilingual User-defined Keyword Spotting](https://mlsanity.com/paper/2406.07310/) (2024)
- [Text-Aware Adapter for Few-Shot Keyword Spotting](https://mlsanity.com/paper/2412.18142/) (2024)
- [Text-Based Detection of On-Hold Scripts in Contact Center Calls](https://mlsanity.com/paper/2407.09849/) (2024)
- [SLiCK: Exploiting Subsequences for Length-Constrained Keyword Spotting](https://mlsanity.com/paper/2409.09067/) (2024)
- [Keyword Spotting with Hyper-Matched Filters for Small Footprint Devices](https://mlsanity.com/paper/2508.04857/) (2025)
- [ProKWS: Personalized Keyword Spotting via Collaborative Learning of Phonemes and Prosody](https://mlsanity.com/paper/2603.18024/) (2026)

## Abstract (arXiv)

Open vocabulary keyword spotting is a crucial and challenging task in automatic speech recognition (ASR) that focuses on detecting user-defined keywords within a spoken utterance. Keyword spotting methods commonly map the audio utterance and keyword into a joint embedding space to obtain some affinity score. In this work, we propose AdaKWS, a novel method for keyword spotting in which a text encoder is trained to output keyword-conditioned normalization parameters. These parameters are used to process the auditory input. We provide an extensive evaluation using challenging and diverse multi-lingual benchmarks and show significant improvements over recent keyword spotting and ASR baselines. Furthermore, we study the effectiveness of our approach on low-resource languages that were unseen during the training. The results demonstrate a substantial performance improvement compared to baseline methods.