# Generative Testing of Automated Speech Recognition Systems

arXiv: https://arxiv.org/abs/2607.09833 · HTML: https://mlsanity.com/paper/2607.09833/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 57 references that are in the map.
- First posted on arXiv 10 Jul 2026.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis](https://mlsanity.com/paper/2010.05646/) (2020)
- [Tacotron: Towards End-to-End Speech Synthesis](https://mlsanity.com/paper/1703.10135/) (2017)
- [UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022](https://mlsanity.com/paper/2204.02152/) (2022)
- [Audio Adversarial Examples: Targeted Attacks on Speech-to-Text](https://mlsanity.com/paper/1801.01944/) (2018)
- [Generating Natural Adversarial Examples](https://mlsanity.com/paper/1710.11342/) (2017)
- [StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Models](https://mlsanity.com/paper/2306.07691/) (2023)
- [Constructing Unrestricted Adversarial Examples with Generative Models](https://mlsanity.com/paper/1805.07894/) (2018)
- [Adversarial Attacks Against Automatic Speech Recognition Systems via Psychoacoustic Hiding](https://mlsanity.com/paper/1808.05665/) (2018)
- [DolphinAtack: Inaudible Voice Commands](https://mlsanity.com/paper/1708.09537/) (2017)
- [Did you hear that? Adversarial Examples Against Automatic Speech Recognition](https://mlsanity.com/paper/1801.00554/) (2018)
- [Targeted Adversarial Examples for Black Box Audio Systems](https://mlsanity.com/paper/1805.07820/) (2018)
- [Enabling Fast and Universal Audio Adversarial Attack Using Generative Model](https://mlsanity.com/paper/2004.12261/) (2020)
- [Black-box Adversarial Attacks on Commercial Speech Platforms with Minimal Information](https://mlsanity.com/paper/2110.09714/) (2021)
- [Metrics and methods for robustness evaluation of neural networks with generative models](https://mlsanity.com/paper/2003.01993/) (2020)
- [Muting Whisper: A Universal Acoustic Adversarial Attack on Speech Foundation Models](https://mlsanity.com/paper/2405.06134/) (2024)
- [ALIF: Low-Cost Adversarial Audio Attacks on Black-Box Speech Platforms using Linguistic Features](https://mlsanity.com/paper/2408.01808/) (2024)
- [Benchmarking Generative AI Models for Deep Learning Test Input Generation](https://mlsanity.com/paper/2412.17652/) (2024)
- [Feature-Aware Test Generation for Deep Learning Models](https://mlsanity.com/paper/2601.14081/) (2026)
- [Neural Predictor for Black-Box Adversarial Attacks on Speech Recognition](https://mlsanity.com/paper/2203.09849/) (2022)

## Abstract (arXiv)

Automatic speech recognition (ASR) systems have achieved high accuracy with transformer-based models, enabling deployment in critical applications. However, they remain vulnerable to adversarial manipulation, particularly in black-box settings where attacks must preserve perceptual naturalness. This work introduces GATAS, a black-box testing approach that generates failure inducing inputs by operating in the phoneme-level latent space of a text- to-speech model. Instead of perturbing waveforms directly, the approach interpolates latent representations to induce transcription errors while remaining within the manifold of natural speech. The attack is formulated as a multi-objective optimization problem balancing semantic divergence and perceptual quality. Our empirical evaluation against both white-box and black-box baselines shows that GATAS achieves a 98% success rate while producing lower distortion and higher perceptual quality, as confirmed by human studies. Despite operating without gradient access, GATAS remains competitive against white-box methods, highlighting that representation and perceptual alignment are more critical than access to model internals. Overall, our results demonstrate that untargeted latent-space optimization enables the efficient generation of realistic and effective test cases for ASR systems.