mlsanityData to 14 Oct 2026

On the Relevance of Phoneme Duration Variability of Synthesized Training Data for Automatic Speech Recognition

arXiv 2310.08132 ↗First posted 12 Oct 2023cs.CL cs.SD eess.AScited by 4

Not on the map. Why?

Abstract (arXiv)

Synthetic data generated by text-to-speech (TTS) systems can be used to improve automatic speech recognition (ASR) systems in low-resource or domain mismatch tasks. It has been shown that TTS-generated outputs still do not have the same qualities as real data. In this work we focus on the temporal structure of synthetic data and its relation to ASR training. By using a novel oracle setup we show how much the degradation of synthetic data quality is influenced by duration modeling in non-autoregressive (NAR) TTS. To get reference phoneme durations we use two common alignment methods, a hidden Markov Gaussian-mixture model (HMM-GMM) aligner and a neural connectionist temporal classification (CTC) aligner. Using a simple algorithm based on random walks we shift phoneme duration distributions of the TTS system closer to real durations, resulting in an improvement of an ASR system using synthetic data in a semi-supervised setting.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 26 of its 43 references that are in the map. First posted on arXiv 12 Oct 2023.

Builds on

26 of 43 references.

References · 26

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  3. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  4. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,380
  5. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 804
  6. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 717
  7. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 649
  8. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 561
  9. 2015Listen, Attend and Spellcited by 543
  10. 2020FastPitch: Parallel Text-to-speech with Pitch Predictioncited by 151
  11. 2019Language Modeling with Deep Transformerscited by 96
  12. 2020Non-Attentive Tacotron: Robust and Controllable Neural TTS Synthesis Including Unsupervised Duration Modelingcited by 65
14 more
  1. 2019Speech Recognition with Augmented Synthesized Speechcited by 60
  2. 2022The VoiceMOS Challenge 2022cited by 57
  3. 2019Generating Synthetic Audio Data for Attention-Based Speech Recognition Systemscited by 47
  4. 2017Exploring Neural Transducers for End-to-End Speech Recognitioncited by 44
  5. 2020Using Synthetic Audio to Improve The Recognition of Out-Of-Vocabulary Words in End-To-End ASR Systemscited by 41
  6. 2018Improved training of end-to-end attention models for speech recognitioncited by 36
  7. 2020You Do Not Need More Data: Improving End-To-End Speech Recognition by Text-To-Speech Data Augmentationcited by 29
  8. 2021Synt++: Utilizing Imperfect Synthetic Data to Improve Speech Recognitioncited by 24
  9. 2016AutoMOS: Learning a non-intrusive assessor of naturalness-of-speechcited by 22
  10. 2016RETURNN: The RWTH Extensible Training framework for Universal Recurrent Neural Networkscited by 16
  11. 2021EAT: Enhanced ASR-TTS for Self-supervised Speech Recognitioncited by 13
  12. 2022Evaluating and reducing the distance between synthetic and real speech distributionscited by 9
  13. 2021Comparing the Benefit of Synthetic Training Data for Various Automatic Speech Recognition Architecturescited by 8
  14. 2021VRAIN-UPV MLLP's system for the Blizzard Challenge 2021cited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 1; 2026: 1.

4 papers.

  1. 2025An Exhaustive Evaluation of TTS- and VC-based Data Augmentation for ASRcited by 5
  2. 2024On the Effect of Purely Synthetic Training Data for Different Automatic Speech Recognition Architecturescited by 4
  3. 2024On the Problem of Text-To-Speech Model Selection for Synthetic Data Generation in Automatic Speech Recognitioncited by 3
  4. 2026Continuous-Time Acoustic Modelling with Neural Controlled Differential Equationscited by 0