# Adversarial training of Keyword Spotting to Minimize TTS Data Overfitting

arXiv: https://arxiv.org/abs/2408.10463 · HTML: https://mlsanity.com/paper/2408.10463/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 28 references that are in the map.
- First posted on arXiv 20 Aug 2024.

## Builds on (most cited first, 17 of 17)

- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [AudioLM: a Language Modeling Approach to Audio Generation](https://mlsanity.com/paper/2209.03143/) (2022)
- [Automated Synthetic-to-Real Generalization](https://mlsanity.com/paper/2007.06965/) (2020)
- [Contrastive Syn-to-Real Generalization](https://mlsanity.com/paper/2104.02290/) (2021)
- [End-to-End Streaming Keyword Spotting](https://mlsanity.com/paper/1812.02802/) (2018)
- [Synt++: Utilizing Imperfect Synthetic Data to Improve Speech Recognition](https://mlsanity.com/paper/2110.11479/) (2021)
- [Speaker-Invariant Training via Adversarial Learning](https://mlsanity.com/paper/1804.00732/) (2018)
- [Training Keyword Spotters with Limited and Synthesized Speech Data](https://mlsanity.com/paper/2002.01322/) (2020)
- [Exploring Speech Enhancement with Generative Adversarial Networks for Robust Speech Recognition](https://mlsanity.com/paper/1711.05747/) (2017)
- [A Cascade Architecture for Keyword Spotting on Mobile Devices](https://mlsanity.com/paper/1712.03603/) (2017)
- [Virtuoso: Massive Multilingual Speech-Text Joint Semi-Supervised Learning for Text-To-Speech](https://mlsanity.com/paper/2210.15447/) (2022)
- [Adversarial Speaker Adaptation](https://mlsanity.com/paper/1904.12407/) (2019)
- [Learning To Detect Keyword Parts And Whole By Smoothed Max Pooling](https://mlsanity.com/paper/2001.09246/) (2020)
- [Efficient Implementation of the Room Simulator for Training Deep Neural Network Acoustic Models](https://mlsanity.com/paper/1712.03439/) (2017)
- [Extending Multilingual Speech Synthesis to 100+ Languages without Transcribed Data](https://mlsanity.com/paper/2402.18932/) (2024)
- [Open vocabulary keyword spotting through transfer learning from speech synthesis](https://mlsanity.com/paper/2404.03914/) (2024)

## Built on by (most cited first, 1 of 1)

- [GraphemeAug: A Systematic Approach to Synthesized Hard Negative Keyword Spotting Examples](https://mlsanity.com/paper/2505.14814/) (2025)

## Abstract (arXiv)

The keyword spotting (KWS) problem requires large amounts of real speech training data to achieve high accuracy across diverse populations. Utilizing large amounts of text-to-speech (TTS) synthesized data can reduce the cost and time associated with KWS development. However, TTS data may contain artifacts not present in real speech, which the KWS model can exploit (overfit), leading to degraded accuracy on real speech. To address this issue, we propose applying an adversarial training method to prevent the KWS model from learning TTS-specific features when trained on large amounts of TTS data. Experimental results demonstrate that KWS model accuracy on real speech data can be improved by up to 12% when adversarial loss is used in addition to the original KWS loss. Surprisingly, we also observed that the adversarial setup improves accuracy by up to 8%, even when trained solely on TTS and real negative speech data, without any real positive examples.