# Synth4Kws: Synthesized Speech for User Defined Keyword Spotting in Low Resource Environments

arXiv: https://arxiv.org/abs/2407.16840 · HTML: https://mlsanity.com/paper/2407.16840/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 25 references that are in the map.
- First posted on arXiv 23 Jul 2024.

## Builds on (most cited first, 16 of 16)

- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition](https://mlsanity.com/paper/1804.03209/) (2018)
- [Generalized End-to-End Loss for Speaker Verification](https://mlsanity.com/paper/1710.10467/) (2017)
- [Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling](https://mlsanity.com/paper/1902.08295/) (2019)
- [MAESTRO: Matched Speech Text Representations through Modality Matching](https://mlsanity.com/paper/2204.03409/) (2022)
- [Streaming keyword spotting on mobile devices](https://mlsanity.com/paper/2005.06720/) (2020)
- [End-to-End Streaming Keyword Spotting](https://mlsanity.com/paper/1812.02802/) (2018)
- [Training Keyword Spotters with Limited and Synthesized Speech Data](https://mlsanity.com/paper/2002.01322/) (2020)
- [Transfer Learning from Speech Synthesis to Voice Conversion with Non-Parallel Training Data](https://mlsanity.com/paper/2009.14399/) (2020)
- [Virtuoso: Massive Multilingual Speech-Text Joint Semi-Supervised Learning for Text-To-Speech](https://mlsanity.com/paper/2210.15447/) (2022)
- [Extending Multilingual Speech Synthesis to 100+ Languages without Transcribed Data](https://mlsanity.com/paper/2402.18932/) (2024)
- [Extreme Encoder Output Frame Rate Reduction: Improving Computational Latencies of Large End-to-End Models](https://mlsanity.com/paper/2402.17184/) (2024)
- [Synth2Aug: Cross-domain speaker recognition with TTS synthesized speech](https://mlsanity.com/paper/2011.11818/) (2020)
- [EfficientWord-Net: An Open Source Hotword Detection Engine based on One-shot Learning](https://mlsanity.com/paper/2111.00379/) (2021)
- [Locale Encoding For Scalable Multilingual Keyword Spotting Models](https://mlsanity.com/paper/2302.12961/) (2023)

## Built on by (most cited first, 2 of 2)

- [Synthetic data enables context-aware bioacoustic sound event detection](https://mlsanity.com/paper/2503.00296/) (2025)
- [GraphemeAug: A Systematic Approach to Synthesized Hard Negative Keyword Spotting Examples](https://mlsanity.com/paper/2505.14814/) (2025)

## Abstract (arXiv)

One of the challenges in developing a high quality custom keyword spotting (KWS) model is the lengthy and expensive process of collecting training data covering a wide range of languages, phrases and speaking styles. We introduce Synth4Kws - a framework to leverage Text to Speech (TTS) synthesized data for custom KWS in different resource settings. With no real data, we found increasing TTS phrase diversity and utterance sampling monotonically improves model performance, as evaluated by EER and AUC metrics over 11k utterances of the speech command dataset. In low resource settings, with 50k real utterances as a baseline, we found using optimal amounts of TTS data can improve EER by 30.1% and AUC by 46.7%. Furthermore, we mix TTS data with varying amounts of real data and interpolate the real data needed to achieve various quality targets. Our experiments are based on English and single word utterances but the findings generalize to i18n languages and other keyword types.