mlsanityData to 14 Oct 2026

Evaluating and reducing the distance between synthetic and real speech distributions

arXiv 2211.16049 ↗First posted 29 Nov 2022eess.AS cs.CL cs.LGcited by 9

Not on the map. Why?

Abstract (arXiv)

While modern Text-to-Speech (TTS) systems can produce natural-sounding speech, they remain unable to reproduce the full diversity found in natural speech data. We consider the distribution of all possible real speech samples that could be generated by these speakers alongside the distribution of all synthetic samples that could be generated for the same set of speakers, using a particular TTS system. We set out to quantify the distance between real and synthetic speech via a range of utterance-level statistics related to properties of the speaker, speech prosody and acoustic environment. Differences in the distribution of these statistics are evaluated using the Wasserstein distance. We reduce these distances by providing ground-truth values at generation time, and quantify the improvements to the overall distribution distance, approximated using an automatic speech recognition system. Our best system achieves a 10% reduction in distribution distance.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 20 of its 34 references that are in the map. First posted on arXiv 29 Nov 2022.

Builds on

20 of 34 references.

References · 20

  1. 2017VoxCeleb: a large-scale speaker identification datasetcited by 732
  2. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 717
  3. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 649
  4. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 561
  5. 2014Deep Speech: Scaling up end-to-end speech recognitioncited by 354
  6. 2017Generalized End-to-End Loss for Speaker Verificationcited by 293
  7. 2021YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyonecited by 278
  8. 2017Soft-DTW: a Differentiable Loss Function for Time-Seriescited by 271
  9. 2022NaturalSpeech: End-to-End Text to Speech Synthesis with Human-Level Qualitycited by 152
  10. 2020FastPitch: Parallel Text-to-speech with Pitch Predictioncited by 151
  11. 2018Hierarchical Generative Modeling for Controllable Speech Synthesiscited by 93
  12. 2019High Fidelity Speech Synthesis with Adversarial Networkscited by 92
8 more
  1. 2019Speech Recognition with Augmented Synthesized Speechcited by 60
  2. 2021VoiceFixer: Toward General Speech Restoration with Neural Vocodercited by 50
  3. 2021Investigating on Incorporating Pretrained and Learnable Speaker Representations for Multi-Speaker Multi-Style Text-to-Speechcited by 37
  4. 2021LightSpeech: Lightweight and Fast Text to Speech with Neural Architecture Searchcited by 28
  5. 2021Synt++: Utilizing Imperfect Synthetic Data to Improve Speech Recognitioncited by 24
  6. 2018Training Neural Speech Recognition Systems with Synthetic Speech Augmentationcited by 21
  7. 2021Speaker Generationcited by 14
  8. 2022TransFusion: Transcribing Speech with Multinomial Diffusioncited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 3; 2025: 1; 2026: 3.

9 papers.

  1. 2024TTSDS -- Text-to-Speech Distribution Scorecited by 14
  2. 2025An Exhaustive Evaluation of TTS- and VC-based Data Augmentation for ASRcited by 5
  3. 2023On the Relevance of Phoneme Duration Variability of Synthesized Training Data for Automatic Speech Recognitioncited by 4
  4. 2023Evaluating Speech Synthesis by Training Recognizers on Synthetic Speechcited by 3
  5. 2024On the Problem of Text-To-Speech Model Selection for Synthetic Data Generation in Automatic Speech Recognitioncited by 3
  6. 2024Beyond Oversmoothing: Evaluating DDPM and MSE for Scalable Speech Synthesis in ASRcited by 0
  7. 2026Understanding Frechet Speech Distance for Synthetic Speech Quality Evaluationcited by 0
  8. 2026ZeSTA: Zero-Shot TTS Augmentation with Domain-Conditioned Training for Data-Efficient Personalized Speech Synthesiscited by 0
  9. 2026Error Rate is Not Enough: Costs of Augmentation in Low-Resource ASRcited by 0