mlsanityData to 14 Oct 2026

An Exhaustive Evaluation of TTS- and VC-based Data Augmentation for ASR

arXiv 2503.08954 ↗First posted 11 Mar 2025eess.AS cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

Augmenting the training data of automatic speech recognition (ASR) systems with synthetic data generated by text-to-speech (TTS) or voice conversion (VC) has gained popularity in recent years. Several works have demonstrated improvements in ASR performance using this augmentation approach. However, because of the lower diversity of synthetic speech, naively combining synthetic and real data often does not yield the best results. In this work, we leverage recently proposed flow-based TTS/VC models allowing greater speech diversity, and assess the respective impact of augmenting various speech attributes on the word error rate (WER) achieved by several ASR models. Pitch augmentation and VC-based speaker augmentation are found to be ineffective in our setup. Jointly augmenting all other attributes reduces the WER of a Conformer-Transducer model by 11% relative on Common Voice and by up to 35% relative on LibriSpeech compared to training on real data only.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 42 of its 66 references that are in the map. First posted on arXiv 11 Mar 2025.

Builds on

42 of 66 references.

References · 42

  1. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  2. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  3. 2016WaveNet: A Generative Model for Raw Audiocited by 1,722
  4. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
  5. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,380
  6. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,154
  7. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 717
  8. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 649
  9. 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 616
  10. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 561
  11. 2021VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretationcited by 463
  12. 2021Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speechcited by 447
30 more
  1. 2015MUSAN: A Music, Speech, and Noise Corpuscited by 425
  2. 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scalecited by 379
  3. 2017Generalized End-to-End Loss for Speaker Verificationcited by 293
  4. 2020Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Searchcited by 273
  5. 2019NeMo: a toolkit for building AI applications using Neural Modulescited by 235
  6. 2019AUTOVC: Zero-Shot Voice Style Transfer with Only Autoencoder Losscited by 192
  7. 2021A Survey on Neural Speech Synthesiscited by 184
  8. 2022NaturalSpeech: End-to-End Text to Speech Synthesis with Human-Level Qualitycited by 152
  9. 2020FastPitch: Parallel Text-to-speech with Pitch Predictioncited by 151
  10. 2020An Overview of Voice Conversion and its Challenges: From Statistical Modeling to Deep Learningcited by 112
  11. 2022FreeVC: Towards High-Quality Text-Free One-Shot Voice Conversioncited by 80
  12. 2021VQMIVC: Vector Quantization and Mutual Information-Based Unsupervised Speech Representation Disentanglement for One-shot Voice Conversioncited by 64
  13. 2021StarGANv2-VC: A Diverse, Unsupervised, Non-parallel Framework for Natural-Sounding Voice Conversioncited by 58
  14. 2021SC-GlowTTS: an Efficient Zero-Shot Multi-Speaker Text-To-Speech Modelcited by 56
  15. 2020Generating diverse and natural text-to-speech samples using a quantized fine-grained VAE and auto-regressive prosody priorcited by 33
  16. 2021Synt++: Utilizing Imperfect Synthetic Data to Improve Speech Recognitioncited by 24
  17. 2019Improving Performance of End-to-End ASR on Numeric Sequencescited by 17
  18. 2020Improving Low Resource Code-switched ASR using Augmented Code-switched TTScited by 15
  19. 2021Voice Conversion Can Improve ASR in Very Low-Resource Settingscited by 14
  20. 2022Text-free non-parallel many-to-many voice conversion using normalising flowscited by 10
  21. 2022ASR data augmentation in low-resource settings using cross-lingual multi-speaker TTS and cross-lingual voice conversioncited by 10
  22. 2022Data Augmentation for Low-Resource Quechua ASR Improvementcited by 9
  23. 2022Evaluating and reducing the distance between synthetic and real speech distributionscited by 9
  24. 2022When Is TTS Augmentation Through a Pivot Language Useful?cited by 8
  25. 2022Can we use Common Voice to train a Multi-Speaker TTS system?cited by 6
  26. 2023On the Relevance of Phoneme Duration Variability of Synthesized Training Data for Automatic Speech Recognitioncited by 4
  27. 2023Varianceflow: High-Quality and Controllable Text-to-Speech using Variance Information via Normalizing Flowcited by 3
  28. 2023Stochastic Pitch Prediction Improves the Diversity and Naturalness of Speech in Glow-TTScited by 3
  29. 2023Comparing normalizing flows and diffusion models for prosody and acoustic modelling in text-to-speechcited by 2
  30. 2023AE-Flow: AutoEncoder Normalizing Flowcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026Enhancing ASR Performance in the Medical Domain for Dravidian Languagescited by 0
  2. 2026Adding Robust Code-Switching Capabilities to High Performance Multilingual ASRcited by 0
  3. 2026Synthetic Speech, Real Signal: Paralinguistic Preservation and Cross-Lingual Augmentation via Voice Cloningcited by 0
  4. 2026Phoneme-guided TTS augmentation for ASR: A unified pipeline and multilingual evaluationcited by 0
  5. 2026A Practical Recipe for Semi-Supervised Federated ASR: Online Pseudo-Labels with Server Update Stabilizationcited by 0