mlsanityData to 7 Oct 2026

On the Problem of Text-To-Speech Model Selection for Synthetic Data Generation in Automatic Speech Recognition

arXiv 2407.21476 ↗First posted 31 Jul 2024cs.CL cs.LG cs.SDcited by 3

Not on the map. Why?

Abstract (arXiv)

The rapid development of neural text-to-speech (TTS) systems enabled its usage in other areas of natural language processing such as automatic speech recognition (ASR) or spoken language translation (SLT). Due to the large number of different TTS architectures and their extensions, selecting which TTS systems to use for synthetic data creation is not an easy task. We use the comparison of five different TTS decoder architectures in the scope of synthetic data generation to show the impact on CTC-based speech recognition training. We compare the recognition results to computable metrics like NISQA MOS and intelligibility, finding that there are no clear relations to the ASR performance. We also observe that for data generation auto-regressive decoding performs better than non-autoregressive decoding, and propose an approach to quantify TTS generalization capabilities.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 23 of its 33 references that are in the map. First posted on arXiv 31 Jul 2024.

Builds on

23 of 33 references.

References · 23

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  3. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
  4. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
  5. 2015Listen, Attend and Spellcited by 542
  6. 2021Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speechcited by 445
  7. 2024Seed-TTS: A Family of High-Quality Versatile Speech Generation Modelscited by 293
  8. 2020Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Searchcited by 269
  9. 2021A Survey on Neural Speech Synthesiscited by 182
  10. 2020Non-Attentive Tacotron: Robust and Controllable Neural TTS Synthesis Including Unsupervised Duration Modelingcited by 65
  11. 2019ESPnet-TTS: Unified, Reproducible, and Integratable Open Source End-to-End Text-to-Speech Toolkitcited by 60
  12. 2022AdaSpeech 4: Adaptive Text to Speech in Zero-Shot Scenarioscited by 41
11 more
  1. 2021Deep Learning Based Assessment of Synthetic Speech Naturalnesscited by 35
  2. 2020You Do Not Need More Data: Improving End-To-End Speech Recognition by Text-To-Speech Data Augmentationcited by 29
  3. 2022Flashlight: Enabling Innovation in Tools for Machine Learningcited by 25
  4. 2021Synt++: Utilizing Imperfect Synthetic Data to Improve Speech Recognitioncited by 24
  5. 2020Phoneme Based Neural Transducer for Large Vocabulary Speech Recognitioncited by 20
  6. 2021EAT: Enhanced ASR-TTS for Self-supervised Speech Recognitioncited by 13
  7. 2018End-to-End Feedback Loss in Speech Chain Framework via Straight-Through Estimatorcited by 11
  8. 2021Comparing the Benefit of Synthetic Training Data for Various Automatic Speech Recognition Architecturescited by 8
  9. 2022Evaluating and reducing the distance between synthetic and real speech distributionscited by 8
  10. 2023On the Relevance of Phoneme Duration Variability of Synthesized Training Data for Automatic Speech Recognitioncited by 4
  11. 2023Comparing normalizing flows and diffusion models for prosody and acoustic modelling in text-to-speechcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 0; 2026: 1.

3 papers.

  1. 2024Enhancing Low-Resource ASR through Versatile TTS: Bridging the Data Gapcited by 11
  2. 2024Beyond Oversmoothing: Evaluating DDPM and MSE for Scalable Speech Synthesis in ASRcited by 0
  3. 2026LuxSQA: Ask Me in Luxembourgish with TTS-Augmented Spoken Question Answeringcited by 0