mlsanityData to 7 Oct 2026

Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech

arXiv 2106.06103 ↗First posted 11 Jun 2021cs.SD eess.AScited by 555

Not on the map. Why?

Cited by 68 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Several recent end-to-end text-to-speech (TTS) models enabling single-stage training and parallel sampling have been proposed, but their sample quality does not match that of two-stage TTS systems. In this work, we present a parallel end-to-end TTS method that generates more natural sounding audio than current two-stage models. Our method adopts variational inference augmented with normalizing flows and an adversarial training process, which improves the expressive power of generative modeling. We also propose a stochastic duration predictor to synthesize speech with diverse rhythms from input text. With the uncertainty modeling over latent variables and the stochastic duration predictor, our method expresses the natural one-to-many relationship in which a text input can be spoken in multiple ways with different pitches and rhythms. A subjective human evaluation (mean opinion score, or MOS) on the LJ Speech, a single speaker dataset, shows that our method outperforms the best publicly available TTS systems and achieves a MOS comparable to ground truth.

Key facts. Cited by 555 papers in the map (data through 7 Oct 2026). First posted on arXiv 11 Jun 2021.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 63; 2023: 115; 2024: 156; 2025: 120; 2026: 97.

555 papers.

Sort:

By area:

  1. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 737
  2. 2023High-Fidelity Audio Compression with Improved RVQGANcited by 423
  3. 2023Scaling Speech Technology to 1,000+ Languagescited by 401
  4. 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scalecited by 373
  5. 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matchingcited by 353
  6. 2021YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyonecited by 277
  7. 2023NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizerscited by 233
  8. 2024NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Modelscited by 222
  9. 2021A Survey on Neural Speech Synthesiscited by 182
  10. 2024XTTS: a Massively Multilingual Zero-Shot Text-to-Speech Modelcited by 174
  11. 2024LLaMA-Omni: Seamless Speech Interaction with Large Language Modelscited by 169
  12. 2024MaskGCT: Zero-Shot Text-to-Speech with Masked Generative Codec Transformercited by 159
38 more
  1. 2022NaturalSpeech: End-to-End Text to Speech Synthesis with Human-Level Qualitycited by 151
  2. 2023StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Modelscited by 148
  3. 2022ProDiff: Progressive Fast Diffusion Model For High-Quality Text-to-Speechcited by 147
  4. 2023Matcha-TTS: A fast TTS architecture with conditional flow matchingcited by 142
  5. 2022FastDiff: A Fast Conditional Diffusion Model for High-Quality Speech Synthesiscited by 136
  6. 2024VoiceCraft: Zero-Shot Speech Editing and Text-to-Speech in the Wildcited by 118
  7. 2024ASVspoof 5: Crowdsourced Speech Data, Deepfakes, and Adversarial Attacks at Scalecited by 88
  8. 2024Recent Advances in Speech Language Models: A Surveycited by 85
  9. 2021Guided-TTS: A Diffusion Model for Text-to-Speech via Classifier Guidancecited by 84
  10. 2022FreeVC: Towards High-Quality Text-Free One-Shot Voice Conversioncited by 79
  11. 2023AV-Deepfake1M: A Large-Scale LLM-Driven Audio-Visual Deepfake Datasetcited by 79
  12. 2024BASE TTS: Lessons from building a billion-parameter Text-to-Speech model on 100K hours of datacited by 74
  13. 2024Detecting Multimedia Generated by Large AI Models: A Surveycited by 73
  14. 2023Voxtlm: unified decoder-only models for consolidating speech recognition/synthesis and speech/text continuation taskscited by 71
  15. 2024Proactive Detection of Voice Cloning with Localized Watermarkingcited by 67
  16. 2023Generative Pre-training for Speech with Flow Matchingcited by 55
  17. 2023Mega-TTS: Zero-Shot Text-to-Speech at Scale with Intrinsic Inductive Biascited by 51
  18. 2023DINOISER: Diffused Conditional Sequence Learning by Manipulating Noisescited by 50
  19. 2024ELLA-V: Stable Neural Codec Language Modeling with Alignment-guided Sequence Reorderingcited by 50
  20. 2023VoiceFlow: Efficient Text-to-Speech with Rectified Flow Matchingcited by 49
  21. 2025VLAS: Vision-Language-Action Model With Speech Instructions For Customized Robot Manipulationcited by 48
  22. 2023CoMoSpeech: One-Step Speech and Singing Voice Synthesis via Consistency Modelcited by 44
  23. 2024EMOVA: Empowering Language Models to See, Hear and Speak with Vivid Emotionscited by 44
  24. 2022DiffGAN-TTS: High-Fidelity and Efficient Text-to-Speech with Denoising Diffusion GANscited by 43
  25. 2024SemantiCodec: An Ultra Low Bitrate Semantic Audio Codec for General Soundcited by 42
  26. 2023WavMark: Watermarking for Audio Generationcited by 41
  27. 2024A Full-duplex Speech Dialogue Scheme Based On Large Language Modelscited by 40
  28. 2023Diffusion Language Models Can Perform Many Tasks with Scaling and Instruction-Finetuningcited by 39
  29. 2024The VoicePrivacy 2024 Challenge Evaluation Plancited by 39
  30. 2024Zero-shot Voice Conversion with Diffusion Transformerscited by 39
  31. 2025MegaTTS 3: Sparse Alignment Enhanced Latent Diffusion Transformer for Zero-Shot Speech Synthesiscited by 39
  32. 2023DDDM-VC: Decoupled Denoising Diffusion Models with Disentangled Representation and Prior Mixup for Verified Robust Voice Conversioncited by 38
  33. 2022EmoDiff: Intensity Controllable Emotional Text-to-Speech with Soft-Label Guidancecited by 37
  34. 2024SpeechGPT-Gen: Scaling Chain-of-Information Speech Generationcited by 36
  35. 2022Guided-TTS 2: A Diffusion Model for High-quality Adaptive Text-to-Speech with Untranscribed Datacited by 35
  36. 2023HierSpeech++: Bridging the Gap between Semantic and Acoustic Representation of Speech by Hierarchical Variational Inference for Zero-shot Speech Synthesiscited by 35
  37. 2022Grad-StyleSpeech: Any-speaker Adaptive Text-to-Speech Synthesis with Diffusion Modelscited by 34
  38. 2022StyleTTS: A Style-Based Generative Model for Natural and Diverse Text-to-Speech Synthesiscited by 32

50 most cited of 555 All 555 (JSON)