mlsanityData to 7 Oct 2026

End-to-End Adversarial Text-to-Speech

arXiv 2006.03575 ↗First posted 5 Jun 2020cs.SD cs.LG eess.AScited by 77

Not on the map. Why?

Abstract (arXiv)

Modern text-to-speech synthesis pipelines typically involve multiple processing stages, each of which is designed or learnt independently from the rest. In this work, we take on the challenging task of learning to synthesise speech from normalised text or phonemes in an end-to-end manner, resulting in models which operate directly on character or phoneme input sequences and produce raw speech audio outputs. Our proposed generator is feed-forward and thus efficient for both training and inference, using a differentiable alignment scheme based on token length prediction. It learns to produce high fidelity audio through a combination of adversarial feedback and prediction losses constraining the generated audio to roughly match the ground truth in terms of its total duration and mel-spectrogram. To allow the model to capture temporal variation in the generated audio, we employ soft dynamic time warping in the spectrogram-based prediction loss. The resulting model achieves a mean opinion score exceeding 4 on a 5 point scale, which is comparable to the state-of-the-art models relying on multi-stage training and additional supervision.

Key facts. Cited by 77 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 23; 2022: 21; 2023: 24; 2024: 6; 2025: 1; 2026: 1.

77 papers.

Sort:

By area:

  1. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  2. 2021Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speechcited by 445
  3. 2021A Survey on Neural Speech Synthesiscited by 182
  4. 2021Diff-TTS: A Denoising Diffusion Model for Text-to-Speechcited by 156
  5. 2022NaturalSpeech: End-to-End Text to Speech Synthesis with Human-Level Qualitycited by 151
  6. 2023StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Modelscited by 148
  7. 2022ProDiff: Progressive Fast Diffusion Model For High-Quality Text-to-Speechcited by 147
  8. 2022FastDiff: A Fast Conditional Diffusion Model for High-Quality Speech Synthesiscited by 136
  9. 2021Guided-TTS: A Diffusion Model for Text-to-Speech via Classifier Guidancecited by 84
  10. 2021WaveFake: A Data Set to Facilitate Audio Deepfake Detectioncited by 78
  11. 2021BLT: Bidirectional Layout Transformer for Controllable Layout Generationcited by 69
  12. 2023A Survey on Audio Diffusion Models: Text To Speech Synthesis and Enhancement in Generative AIcited by 68
38 more
  1. 2021WaveGrad 2: Iterative Refinement for Text-to-Speech Synthesiscited by 65
  2. 2023Mega-TTS: Zero-Shot Text-to-Speech at Scale with Intrinsic Inductive Biascited by 51
  3. 2022DiffGAN-TTS: High-Fidelity and Efficient Text-to-Speech with Denoising Diffusion GANscited by 43
  4. 2022Guided-TTS 2: A Diffusion Model for High-quality Adaptive Text-to-Speech with Untranscribed Datacited by 35
  5. 2023E3 TTS: Easy End-to-End Diffusion-based Text to Speechcited by 30
  6. 2021End-to-End Video-To-Speech Synthesis using Generative Adversarial Networkscited by 27
  7. 2023Make-A-Voice: Unified Voice Synthesis With Discrete Representationcited by 27
  8. 2022JETS: Jointly Training FastSpeech2 and HiFi-GAN for End to End Text to Speechcited by 26
  9. 2021Universal Neural Vocoding with Parallel WaveNetcited by 25
  10. 2023VITS2: Improving Quality and Efficiency of Single-Stage Text-to-Speech with Adversarial Learning and Architecture Designcited by 25
  11. 2021AligNART: Non-autoregressive Neural Machine Translation by Jointly Learning to Estimate Alignment and Translatecited by 24
  12. 2023ViT-TTS: Visual Text-to-Speech with Scalable Diffusion Transformercited by 24
  13. 2022Uncertainty-DTW for Time Series and Sequencescited by 21
  14. 2022Language-Agnostic Meta-Learning for Low-Resource Text-to-Speech with Articulatory Featurescited by 20
  15. 2021AnyoneNet: Synchronized Speech and Talking Head Generation for Arbitrary Personcited by 17
  16. 2022WavThruVec: Latent speech representation as intermediate features for neural speech synthesiscited by 17
  17. 2023The Age of Synthetic Realities: Challenges and Opportunitiescited by 15
  18. 2024CM-TTS: Enhancing Real Time Text-to-Speech Synthesis Efficiency through Weighted Samplers and Consistency Modelscited by 13
  19. 2024SSDM: Scalable Speech Dysfluency Modelingcited by 13
  20. 2021VARA-TTS: Non-Autoregressive Text-to-Speech Synthesis based on Very Deep VAE with Residual Attentioncited by 12
  21. 2023RMSSinger: Realistic-Music-Score based Singing Voice Synthesiscited by 12
  22. 2021Style Equalization: Unsupervised Learning of Controllable Generative Sequence Modelscited by 11
  23. 2023DiffAR: Denoising Diffusion Autoregressive Model for Raw Speech Waveform Generationcited by 11
  24. 2021On-device neural speech synthesiscited by 9
  25. 2021Hierarchical prosody modeling and control in non-autoregressive parallel neural TTScited by 9
  26. 2021Emphasis control for parallel neural TTScited by 9
  27. 2022Audio representations for deep learning in sound synthesis: A reviewcited by 9
  28. 2023HiddenSinger: High-Quality Singing Voice Synthesis via Neural Audio Codec and Latent Diffusion Modelscited by 9
  29. 2021Neural HMMs are all you need (for high-quality attention-free TTS)cited by 8
  30. 2022FastLTS: Non-Autoregressive End-to-End Unconstrained Lip-to-Speech Synthesiscited by 8
  31. 2024RapVerse: Coherent Vocals and Whole-Body Motions Generations from Textcited by 7
  32. 2021Review of end-to-end speech synthesis technology based on deep learningcited by 6
  33. 2022Period VITS: Variational Inference with Explicit Pitch Modeling for End-to-end Emotional Speech Synthesiscited by 6
  34. 2023The IMS Toucan System for the Blizzard Challenge 2023cited by 6
  35. 2024Improving Generalization for AI-Synthesized Voice Detectioncited by 6
  36. 2023PauseSpeech: Natural Speech Synthesis via Pre-trained Language Model and Pause-based Prosody Modelingcited by 5
  37. 2024A Survey of Deep Learning Audio Generation Methodscited by 5
  38. 2023Unsupervised Data Selection for TTS: Using Arabic Broadcast News as a Case Studycited by 4

50 most cited of 77 All 77 (JSON)