# You Do Not Need More Data: Improving End-To-End Speech Recognition by Text-To-Speech Data Augmentation

arXiv: https://arxiv.org/abs/2005.07157 · HTML: https://mlsanity.com/paper/2005.07157/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 May 2020.

## Built on by (most cited first, 29 of 29)

- [Self-Supervised Speech Representation Learning: A Review](https://mlsanity.com/paper/2205.10643/) (2022)
- [Voice Conversion Can Improve ASR in Very Low-Resource Settings](https://mlsanity.com/paper/2111.02674/) (2021)
- [Enhancing Low-Resource ASR through Versatile TTS: Bridging the Data Gap](https://mlsanity.com/paper/2410.16726/) (2024)
- [End-to-end model for named entity recognition from speech without paired training data](https://mlsanity.com/paper/2204.00803/) (2022)
- [Text-only domain adaptation for end-to-end ASR using integrated text-to-mel-spectrogram generator](https://mlsanity.com/paper/2302.14036/) (2023)
- [ASR data augmentation in low-resource settings using cross-lingual multi-speaker TTS and cross-lingual voice conversion](https://mlsanity.com/paper/2204.00618/) (2022)
- [On-the-Fly Aligned Data Augmentation for Sequence-to-Sequence ASR](https://mlsanity.com/paper/2104.01393/) (2021)
- [Comparing the Benefit of Synthetic Training Data for Various Automatic Speech Recognition Architectures](https://mlsanity.com/paper/2104.05379/) (2021)
- [Learning Speech Emotion Representations in the Quaternion Domain](https://mlsanity.com/paper/2204.02385/) (2022)
- [A Simple Baseline for Domain Adaptation in End to End ASR Systems Using Synthetic Data](https://mlsanity.com/paper/2206.13240/) (2022)
- [Back Transcription as a Method for Evaluating Robustness of Natural Language Understanding Models to Speech Recognition Errors](https://mlsanity.com/paper/2310.16609/) (2023)
- [OLISIA: a Cascade System for Spoken Dialogue State Tracking](https://mlsanity.com/paper/2304.11073/) (2023)
- [On the Relevance of Phoneme Duration Variability of Synthesized Training Data for Automatic Speech Recognition](https://mlsanity.com/paper/2310.08132/) (2023)
- [On the Effect of Purely Synthetic Training Data for Different Automatic Speech Recognition Architectures](https://mlsanity.com/paper/2407.17997/) (2024)
- [On the Problem of Text-To-Speech Model Selection for Synthetic Data Generation in Automatic Speech Recognition](https://mlsanity.com/paper/2407.21476/) (2024)
- [Dynamic Acoustic Unit Augmentation With BPE-Dropout for Low-Resource End-to-End Speech Recognition](https://mlsanity.com/paper/2103.07186/) (2021)
- [Data Incubation -- Synthesizing Missing Data for Handwriting Recognition](https://mlsanity.com/paper/2110.07040/) (2021)
- [Towards Robust Waveform-Based Acoustic Models](https://mlsanity.com/paper/2110.08634/) (2021)
- [Emphasizing Unseen Words: New Vocabulary Acquisition for End-to-End Speech Recognition](https://mlsanity.com/paper/2302.09723/) (2023)
- [Contrastive Learning from Synthetic Audio Doppelg"angers](https://mlsanity.com/paper/2406.05923/) (2024)
- [Label-Synchronous Speech-to-Text Alignment for ASR Using Forward and Backward Transformers](https://mlsanity.com/paper/2104.10328/) (2021)
- [JoeyS2T: Minimalistic Speech-to-Text Modeling with JoeyNMT](https://mlsanity.com/paper/2210.02545/) (2022)
- [MAC: A unified framework boosting low resource automatic speech recognition](https://mlsanity.com/paper/2302.03498/) (2023)
- [Improving End-to-End Speech Processing by Efficient Text Data Utilization with Latent Synthesis](https://mlsanity.com/paper/2310.05374/) (2023)
- [Reproducibility and Geometric Intrinsic Dimensionality: An Investigation on Graph Neural Network Research](https://mlsanity.com/paper/2403.08438/) (2024)
- [Initial Decoding with Minimally Augmented Language Model for Improved Lattice Rescoring in Low Resource ASR](https://mlsanity.com/paper/2403.10937/) (2024)
- [Enhancing Synthetic Training Data for Speech Commands: From ASR-Based Filtering to Domain Adaptation in SSL Latent Space](https://mlsanity.com/paper/2409.12745/) (2024)
- [Development and multi-center evaluation of domain-adapted speech recognition for human-AI teaming in real-world gastrointestinal endoscopy](https://mlsanity.com/paper/2604.01705/) (2026)
- [Synthetic Speech, Real Signal: Paralinguistic Preservation and Cross-Lingual Augmentation via Voice Cloning](https://mlsanity.com/paper/2607.22304/) (2026)

## Abstract (arXiv)

Data augmentation is one of the most effective ways to make end-to-end automatic speech recognition (ASR) perform close to the conventional hybrid approach, especially when dealing with low-resource tasks. Using recent advances in speech synthesis (text-to-speech, or TTS), we build our TTS system on an ASR training database and then extend the data with synthesized speech to train a recognition model. We argue that, when the training data amount is relatively low, this approach can allow an end-to-end model to reach hybrid systems' quality. For an artificial low-to-medium-resource setup, we compare the proposed augmentation with the semi-supervised learning technique. We also investigate the influence of vocoder usage on final ASR performance by comparing Griffin-Lim algorithm with our modified LPCNet. When applied with an external language model, our approach outperforms a semi-supervised setup for LibriSpeech test-clean and only 33% worse than a comparable supervised setup. Our system establishes a competitive result for end-to-end ASR trained on LibriSpeech train-clean-100 set with WER 4.3% for test-clean and 13.5% for test-other.