# Text-aware Speech Separation for Multi-talker Keyword Spotting

arXiv: https://arxiv.org/abs/2406.12447 · HTML: https://mlsanity.com/paper/2406.12447/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jun 2024.

## Built on by (most cited first, 3 of 3)

- [Text-Aware Adapter for Few-Shot Keyword Spotting](https://mlsanity.com/paper/2412.18142/) (2024)
- [Adversarial Deep Metric Learning for Cross-Modal Audio-Text Alignment in Open-Vocabulary Keyword Spotting](https://mlsanity.com/paper/2505.16735/) (2025)
- [MATE: Matryoshka Audio-Text Embeddings for Open-Vocabulary Keyword Spotting](https://mlsanity.com/paper/2601.14012/) (2026)

## Abstract (arXiv)

For noisy environments, ensuring the robustness of keyword spotting (KWS) systems is essential. While much research has focused on noisy KWS, less attention has been paid to multi-talker mixed speech scenarios. Unlike the usual cocktail party problem where multi-talker speech is separated using speaker clues, the key challenge here is to extract the target speech for KWS based on text clues. To address it, this paper proposes a novel Text-aware Permutation Determinization Training method for multi-talker KWS with a clue-based Speech Separation front-end (TPDT-SS). Our research highlights the critical role of SS front-ends and shows that incorporating keyword-specific clues into these models can greatly enhance the effectiveness. TPDT-SS shows remarkable success in addressing permutation problems in mixed keyword speech, thereby greatly boosting the performance of the backend. Additionally, fine-tuning our system on unseen mixed speech results in further performance improvement.