# QbyE-MLPMixer: Query-by-Example Open-Vocabulary Keyword Spotting using MLPMixer

arXiv: https://arxiv.org/abs/2206.13231 · HTML: https://mlsanity.com/paper/2206.13231/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 37 references that are in the map.
- First posted on arXiv 23 Jun 2022.

## Builds on (most cited first, 20 of 20)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognition](https://mlsanity.com/paper/1912.10211/) (2019)
- [Pay Attention to MLPs](https://mlsanity.com/paper/2105.08050/) (2021)
- [ResMLP: Feedforward networks for image classification with data-efficient training](https://mlsanity.com/paper/2105.03404/) (2021)
- [A scalable noisy speech dataset and online subjective test framework](https://mlsanity.com/paper/1909.08050/) (2019)
- [Rethinking CNN Models for Audio Classification](https://mlsanity.com/paper/2007.11154/) (2020)
- [RepMLP: Re-parameterizing Convolutions into Fully-connected Layers for Image Recognition](https://mlsanity.com/paper/2105.01883/) (2021)
- [Discriminative Acoustic Word Embeddings: Recurrent Neural Network-Based Approaches](https://mlsanity.com/paper/1611.02550/) (2016)
- [Query-by-Example Keyword Spotting system using Multi-head Attention and Softtriple Loss](https://mlsanity.com/paper/2102.07061/) (2021)
- [Streaming Small-Footprint Keyword Spotting using Sequence-to-Sequence Models](https://mlsanity.com/paper/1710.09617/) (2017)
- [DONUT: CTC-based Query-by-Example Keyword Spotting](https://mlsanity.com/paper/1811.10736/) (2018)
- [Wake Word Detection with Streaming Transformers](https://mlsanity.com/paper/2102.04488/) (2021)
- [Small-Footprint Open-Vocabulary Keyword Spotting with Quantized LSTM Networks](https://mlsanity.com/paper/2002.10851/) (2020)
- [Hybrid Transformer/CTC Networks for Hardware Efficient Voice Triggering](https://mlsanity.com/paper/2008.02323/) (2020)
- [Howl: A Deployed, Open-Source Wake Word Detection System](https://mlsanity.com/paper/2008.09606/) (2020)

## Built on by (most cited first, 4 of 4)

- [Few-Shot Open-Set Learning for On-Device Customization of KeyWord Spotting Systems](https://mlsanity.com/paper/2306.02161/) (2023)
- [Latent Phrase Matching for Dysarthric Speech](https://mlsanity.com/paper/2306.05446/) (2023)
- [Query-by-Example Keyword Spotting Using Spectral-Temporal Graph Attentive Pooling and Multi-Task Learning](https://mlsanity.com/paper/2409.00099/) (2024)
- [PromptKWS: A Novel Prompt-Guided Open-Vocabulary Keyword Spotting Framework](https://mlsanity.com/paper/2608.28640/) (2026)

## Abstract (arXiv)

Current keyword spotting systems are typically trained with a large amount of pre-defined keywords. Recognizing keywords in an open-vocabulary setting is essential for personalizing smart device interaction. Towards this goal, we propose a pure MLP-based neural network that is based on MLPMixer - an MLP model architecture that effectively replaces the attention mechanism in Vision Transformers. We investigate different ways of adapting the MLPMixer architecture to the QbyE open-vocabulary keyword spotting task. Comparisons with the state-of-the-art RNN and CNN models show that our method achieves better performance in challenging situations (10dB and 6dB environments) on both the publicly available Hey-Snips dataset and a larger scale internal dataset with 400 speakers. Our proposed model also has a smaller number of parameters and MACs compared to the baseline models.