mlsanityData to 7 Oct 2026

WaveGrad 2: Iterative Refinement for Text-to-Speech Synthesis

arXiv 2106.09660 ↗First posted 17 Jun 2021eess.AS cs.LG cs.SDcited by 65

Not on the map. Why?

Cited by 50 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

This paper introduces WaveGrad 2, a non-autoregressive generative model for text-to-speech synthesis. WaveGrad 2 is trained to estimate the gradient of the log conditional density of the waveform given a phoneme sequence. The model takes an input phoneme sequence, and through an iterative refinement process, generates an audio waveform. This contrasts to the original WaveGrad vocoder which conditions on mel-spectrogram features, generated by a separate model. The iterative refinement process starts from Gaussian noise, and through a series of refinement steps (e.g., 50 steps), progressively recovers the audio sequence. WaveGrad 2 offers a natural way to trade-off between inference speed and sample quality, through adjusting the number of refinement steps. Experiments show that the model can generate high fidelity audio, approaching the performance of a state-of-the-art neural TTS system. We also report various ablation studies over different model configurations. Audio samples are available at https://wavegrad.github.io/v2.

Key facts. Cited by 65 papers in the map (data through 7 Oct 2026). Builds on 28 of its 35 references that are in the map. First posted on arXiv 17 Jun 2021.

Builds on

28 of 35 references.

References · 28

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  4. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  5. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
  6. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
  7. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
  8. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 620
  9. 2017Tacotron: Towards End-to-End Speech Synthesiscited by 487
  10. 2017PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modificationscited by 415
  11. 2019MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesiscited by 351
  12. 2019Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogramcited by 337
16 more
  1. 2018WaveGlow: A Flow-based Generative Network for Speech Synthesiscited by 308
  2. 2018Adversarial Audio Synthesiscited by 222
  3. 2017Parallel WaveNet: Fast High-Fidelity Speech Synthesiscited by 214
  4. 2019GANSynth: Adversarial Neural Audio Synthesiscited by 114
  5. 2019High Fidelity Speech Synthesis with Adversarial Networkscited by 90
  6. 2020End-to-End Adversarial Text-to-Speechcited by 77
  7. 2018ClariNet: Parallel Wave Generation in End-to-End Text-to-Speechcited by 70
  8. 2020Non-Attentive Tacotron: Robust and Controllable Neural TTS Synthesis Including Unsupervised Duration Modelingcited by 65
  9. 2020Multi-band MelGAN: Faster Waveform Generation for High-Quality Text-to-Speechcited by 63
  10. 2016Zoneout: Regularizing RNNs by Randomly Preserving Hidden Activationscited by 54
  11. 2018FloWaveNet : A Generative Flow for Raw Audiocited by 54
  12. 2020Wave-Tacotron: Spectrogram-free end-to-end text-to-speech synthesiscited by 38
  13. 2019DurIAN: Duration Informed Attention Network For Multimodal Synthesiscited by 33
  14. 2020VocGAN: A High-Fidelity Real-time Vocoder with a Hierarchically-nested Adversarial Networkcited by 26
  15. 2020HooliGAN: Robust, High Quality Neural Vocodingcited by 7
  16. 2020WaveNODE: A Continuous Normalizing Flow for Speech Synthesiscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 12; 2023: 27; 2024: 11; 2025: 7; 2026: 2.

65 papers.

Sort:

By area:

  1. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,961
  2. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  3. 2021Palette: Image-to-Image Diffusion Modelscited by 1,296
  4. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Modelscited by 924
  5. 2022LION: Latent Point Diffusion Models for 3D Shape Generationcited by 552
  6. 2023Understanding Diffusion Objectives as the ELBO with Simple Data Augmentationcited by 293
  7. 2021Deblurring via Stochastic Refinementcited by 248
  8. 2023DIFUSCO: Graph-based Diffusion Solvers for Combinatorial Optimizationcited by 216
  9. 2022Learning Fast Samplers for Diffusion Models by Differentiating Through Sample Qualitycited by 181
  10. 2023Negative-prompt Inversion: Fast Image Inversion for Editing with Text-guided Diffusion Modelscited by 172
  11. 2022Generative Modelling With Inverse Heat Dissipationcited by 164
  12. 2023StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Modelscited by 148
38 more
  1. 2023Contrastive Energy Prediction for Exact Energy-Guided Diffusion Sampling in Offline Reinforcement Learningcited by 145
  2. 2023Matcha-TTS: A fast TTS architecture with conditional flow matchingcited by 142
  3. 2022FastDiff: A Fast Conditional Diffusion Model for High-Quality Speech Synthesiscited by 136
  4. 2022GENIE: Higher-Order Denoising Diffusion Solverscited by 127
  5. 2025Flow Matching Policy Gradientscited by 86
  6. 2021Guided-TTS: A Diffusion Model for Text-to-Speech via Classifier Guidancecited by 84
  7. 2023A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need?cited by 84
  8. 2023VideoGen: A Reference-Guided Latent Diffusion Approach for High Definition Text-to-Video Generationcited by 74
  9. 2023A Survey on Audio Diffusion Models: Text To Speech Synthesis and Enhancement in Generative AIcited by 68
  10. 2022Guided-TTS 2: A Diffusion Model for High-quality Adaptive Text-to-Speech with Untranscribed Datacited by 35
  11. 2023SEEDS: Exponential SDE Solvers for Fast High-Quality Sampling from Diffusion Modelscited by 35
  12. 2023E3 TTS: Easy End-to-End Diffusion-based Text to Speechcited by 30
  13. 2023GECCO: Geometrically-Conditioned Point Diffusion Modelscited by 24
  14. 2021ESPnet2-TTS: Extending the Edge of TTS Researchcited by 22
  15. 2023DiffV2S: Diffusion-based Video-to-Speech Synthesis with Vision-guided Speaker Embeddingcited by 22
  16. 2022Talking Face Generation with Multilingual TTScited by 18
  17. 2023The Ethical Implications of Generative Audio Models: A Systematic Literature Reviewcited by 18
  18. 2024SimpleSpeech 2: Towards Simple and Efficient Text-to-Speech with Flow-based Scalar Latent Transformer Diffusion Modelscited by 17
  19. 2023Separate And Diffuse: Using a Pretrained Diffusion Model for Improving Source Separationcited by 14
  20. 2023DPM-OT: A New Diffusion Probabilistic Model Based on Optimal Transportcited by 14
  21. 2023DiffVoice: Text-to-Speech with Latent Diffusioncited by 13
  22. 2023Diff-TTSG: Denoising probabilistic integrated speech and gesture synthesiscited by 13
  23. 2023DiffAR: Denoising Diffusion Autoregressive Model for Raw Speech Waveform Generationcited by 11
  24. 2023HiddenSinger: High-Quality Singing Voice Synthesis via Neural Audio Codec and Latent Diffusion Modelscited by 9
  25. 2024Multi-modal Generative AI: Multi-modal LLMs, Diffusions, and the Unificationcited by 8
  26. 2024Diffuse or Confuse: A Diffusion Deepfake Speech Datasetcited by 8
  27. 2024DLPO: Diffusion Model Loss-Guided Reinforcement Learning for Fine-Tuning Text-to-Speech Diffusion Modelscited by 7
  28. 2023Dual Diffusion Architecture for Fisheye Image Rectification: Synthetic-to-Real Generalizationcited by 5
  29. 2023Nested Diffusion Processes for Anytime Image Generationcited by 5
  30. 2023Lightweight Diffusion Models with Distillation-Based Block Neural Architecture Searchcited by 4
  31. 2024GVDIFF: Grounded Text-to-Video Generation with Diffusion Modelscited by 4
  32. 2024WebRPG: Automatic Web Rendering Parameters Generation for Visual Presentationcited by 4
  33. 2024DiffSSD: A Diffusion-Based Dataset For Speech Forensicscited by 4
  34. 2025ItDPDM: Information-Theoretic Discrete Poisson Diffusion Modelcited by 4
  35. 2021Quasi-Taylor Samplers for Diffusion Generative Models based on Ideal Derivativescited by 3
  36. 2023iSTFTNet2: Faster and More Lightweight iSTFT-Based Neural Vocoder Using 1D-2D CNNcited by 3
  37. 2024On the Semantic Latent Space of Diffusion-Based Text-to-Speech Modelscited by 3
  38. 2025Fine-Tuning Text-to-Speech Diffusion Models Using Reinforcement Learning with Human Feedbackcited by 3

50 most cited of 65 All 65 (JSON)