# MATE: Matryoshka Audio-Text Embeddings for Open-Vocabulary Keyword Spotting

arXiv: https://arxiv.org/abs/2601.14012 · HTML: https://mlsanity.com/paper/2601.14012/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 30 references that are in the map.
- First posted on arXiv 20 Jan 2026.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification](https://mlsanity.com/paper/2005.07143/) (2020)
- [MUSAN: A Music, Speech, and Noise Corpus](https://mlsanity.com/paper/1510.08484/) (2015)
- [Multi-Similarity Loss with General Pair Weighting for Deep Metric Learning](https://mlsanity.com/paper/1904.06627/) (2019)
- [Matryoshka Representation Learning](https://mlsanity.com/paper/2205.13147/) (2022)
- [Comparing Kullback-Leibler Divergence and Mean Squared Error Loss in Knowledge Distillation](https://mlsanity.com/paper/2105.08919/) (2021)
- [Deep Residual Learning for Small-Footprint Keyword Spotting](https://mlsanity.com/paper/1710.10361/) (2017)
- [Query-by-Example Keyword Spotting system using Multi-head Attention and Softtriple Loss](https://mlsanity.com/paper/2102.07061/) (2021)
- [Multi-view Recurrent Neural Acoustic Word Embeddings](https://mlsanity.com/paper/1611.04496/) (2016)
- [Learning Audio-Text Agreement for Open-vocabulary Keyword Spotting](https://mlsanity.com/paper/2206.15400/) (2022)
- [Matching Latent Encoding for Audio-Text based Keyword Spotting](https://mlsanity.com/paper/2306.05245/) (2023)
- [PhonMatchNet: Phoneme-Guided Zero-Shot Keyword Spotting for User-Defined Keywords](https://mlsanity.com/paper/2308.16511/) (2023)
- [Phoneme-Level Contrastive Learning for User-Defined Keyword Spotting with Flexible Enrollment](https://mlsanity.com/paper/2412.20805/) (2024)
- [Asymmetric Proxy Loss for Multi-View Acoustic Word Embeddings](https://mlsanity.com/paper/2203.16080/) (2022)
- [AdaMS: Deep Metric Learning with Adaptive Margin and Adaptive Scale for Acoustic Word Discrimination](https://mlsanity.com/paper/2210.14564/) (2022)
- [Relational Proxy Loss for Audio-Text based Keyword Spotting](https://mlsanity.com/paper/2406.05314/) (2024)
- [Text-aware Speech Separation for Multi-talker Keyword Spotting](https://mlsanity.com/paper/2406.12447/) (2024)
- [M-Vec: Matryoshka Speaker Embeddings with Flexible Dimensions](https://mlsanity.com/paper/2409.15782/) (2024)
- [Text-Aware Adapter for Few-Shot Keyword Spotting](https://mlsanity.com/paper/2412.18142/) (2024)
- [Adversarial Deep Metric Learning for Cross-Modal Audio-Text Alignment in Open-Vocabulary Keyword Spotting](https://mlsanity.com/paper/2505.16735/) (2025)

## Abstract (arXiv)

Open-vocabulary keyword spotting (KWS) with text-based enrollment has emerged as a flexible alternative to fixed-phrase triggers. Prior utterance-level matching methods, from an embedding-learning standpoint, learn embeddings at a single fixed dimensionality. We depart from this design and propose Matryoshka Audio-Text Embeddings (MATE), a dual-encoder framework that encodes multiple embedding granularities within a single vector via nested sub-embeddings ("prefixes"). Specifically, we introduce a PCA-guided prefix alignment: PCA-compressed versions of the full text embedding for each prefix size serve as teacher targets to align both audio and text prefixes. This alignment concentrates salient keyword cues in lower-dimensional prefixes, while higher dimensions add detail. MATE is trained with standard deep metric learning objectives for audio-text KWS, and is loss-agnostic. To our knowledge, this is the first application of matryoshka-style embeddings to KWS, achieving state-of-the-art results on WSJ and LibriPhrase without any inference overhead.