mlsanityData to 7 Oct 2026

Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech

arXiv 2105.06337 ↗First posted 13 May 2021cs.LG cs.CL stat.MLcited by 445

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

Recently, denoising diffusion probabilistic models and generative score matching have shown high potential in modelling complex data distributions while stochastic calculus has provided a unified point of view on these techniques allowing for flexible inference schemes. In this paper we introduce Grad-TTS, a novel text-to-speech model with score-based decoder producing mel-spectrograms by gradually transforming noise predicted by encoder and aligned with text input by means of Monotonic Alignment Search. The framework of stochastic differential equations helps us to generalize conventional diffusion probabilistic models to the case of reconstructing data from noise with different parameters and allows to make this reconstruction flexible by explicitly controlling trade-off between sound quality and inference speed. Subjective human evaluation shows that Grad-TTS is competitive with state-of-the-art text-to-speech approaches in terms of Mean Opinion Score. We will make the code publicly available shortly.

Key facts. Cited by 445 papers in the map (data through 7 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 27 of its 36 references that are in the map. First posted on arXiv 13 May 2021.

Builds on

27 of 36 references.

Closest ancestors · 1

Same direction, newest first

  1. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesis21 Sep

Other references · 26

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  5. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  6. 2018Neural Ordinary Differential Equationscited by 3,733
  7. 2017Improved Training of Wasserstein GANscited by 2,223
  8. 2015Variational Inference with Normalizing Flowscited by 2,161
  9. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  10. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
  11. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  12. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,036
14 more
  1. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
  2. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 620
  3. 2019MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesiscited by 351
  4. 2019Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogramcited by 337
  5. 2018WaveGlow: A Flow-based Generative Network for Speech Synthesiscited by 308
  6. 2020Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Searchcited by 269
  7. 2020Permutation Invariant Graph Generation via Score-Based Generative Modelingcited by 263
  8. 2020Learning Gradient Fields for Shape Generationcited by 245
  9. 2017Parallel WaveNet: Fast High-Fidelity Speech Synthesiscited by 214
  10. 2019High Fidelity Speech Synthesis with Adversarial Networkscited by 90
  11. 2020End-to-End Adversarial Text-to-Speechcited by 77
  12. 2020Non-Attentive Tacotron: Robust and Controllable Neural TTS Synthesis Including Unsupervised Duration Modelingcited by 65
  13. 2020Parallel Tacotron: Non-Autoregressive and Controllable TTScited by 49
  14. 2020Diffusion models for Handwriting Generationcited by 25

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 10; 2022: 53; 2023: 123; 2024: 129; 2025: 80; 2026: 50.

445 papers.

Sort:

By area:

  1. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  2. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  3. 2023Consistency Modelscited by 1,778
  4. 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,172
  5. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,107
  6. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 737
  7. 2021Diffusion Schr"odinger Bridge with Applications to Score-Based Generative Modelingcited by 594
  8. 2023AudioLDM: Text-to-Audio Generation with Latent Diffusion Modelscited by 592
  9. 2022LION: Latent Point Diffusion Models for 3D Shape Generationcited by 552
  10. 2022Multimodal Learning with Transformers: A Surveycited by 399
  11. 2023Analyzing and Improving the Training Dynamics of Diffusion Modelscited by 396
  12. 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scalecited by 373
38 more
  1. 2022DiffusionDet: Diffusion Model for Object Detectioncited by 354
  2. 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matchingcited by 353
  3. 2023DeepCache: Accelerating Diffusion Models for Freecited by 346
  4. 2023AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretrainingcited by 343
  5. 2022Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Modelscited by 331
  6. 2021SegDiff: Image Segmentation with Diffusion Probabilistic Modelscited by 279
  7. 2022Diffsound: Discrete Diffusion Model for Text-to-sound Generationcited by 256
  8. 2024Applying Guidance in a Limited Interval Improves Sample and Distribution Quality in Diffusion Modelscited by 240
  9. 2022A Survey on Generative Diffusion Modelcited by 238
  10. 2023NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizerscited by 233
  11. 2023Nearly d-Linear Convergence Bounds for Diffusion Models via Stochastic Localizationcited by 230
  12. 2024NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Modelscited by 222
  13. 2023Guided Motion Diffusion for Controllable Human Motion Synthesiscited by 221
  14. 2022MoFusion: A Framework for Denoising-Diffusion-based Motion Synthesiscited by 218
  15. 2022Guided Conditional Diffusion for Controllable Traffic Simulationcited by 189
  16. 2023One Transformer Fits All Distributions in Multi-Modal Diffusion at Scalecited by 185
  17. 2022Stochastic Trajectory Prediction via Motion Indeterminacy Diffusioncited by 184
  18. 2021A Survey on Neural Speech Synthesiscited by 182
  19. 2022Fast Sampling of Diffusion Models via Operator Learningcited by 173
  20. 2023Noise2Music: Text-conditioned Music Generation with Diffusion Modelscited by 173
  21. 2021On Fast Sampling of Diffusion Probabilistic Modelscited by 170
  22. 2022NaturalSpeech: End-to-End Text to Speech Synthesis with Human-Level Qualitycited by 151
  23. 2023StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Modelscited by 148
  24. 2022ProDiff: Progressive Fast Diffusion Model For High-Quality Text-to-Speechcited by 147
  25. 2023Text-to-Audio Generation using Instruction-Tuned LLM and Latent Diffusion Modelcited by 145
  26. 2023Matcha-TTS: A fast TTS architecture with conditional flow matchingcited by 142
  27. 2023Patch Diffusion: Faster and More Data-Efficient Training of Diffusion Modelscited by 131
  28. 2023DiffusionSat: A Generative Foundation Model for Satellite Imagerycited by 130
  29. 2023Editing Implicit Assumptions in Text-to-Image Diffusion Modelscited by 128
  30. 2022GENIE: Higher-Order Denoising Diffusion Solverscited by 127
  31. 2022Score-Based Generative Models Detect Manifoldscited by 121
  32. 2023TRACT: Denoising Diffusion Models with Transitive Closure Time-Distillationcited by 115
  33. 2023Diffusion Models for Image Restoration and Enhancement: A Comprehensive Surveycited by 108
  34. 2022How to Backdoor Diffusion Models?cited by 102
  35. 2021Diffusion-Based Voice Conversion with Fast Maximum Likelihood Sampling Schemecited by 99
  36. 2024Flow map matching with stochastic interpolants: A mathematical framework for consistency modelscited by 99
  37. 2023Optimizing Diffusion Noise Can Serve As Universal Motion Priorscited by 97
  38. 2023Error Bounds for Flow Matching Methodscited by 88

50 most cited of 445 All 445 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2021Diff-TTS: A Denoising Diffusion Model for Text-to-Speech
  2. 2024Boosting Diffusion Model for Spectrogram Up-sampling in Text-to-speech: An Empirical Study
  3. 2025Discrete Diffusion for Generative Modeling of Text-Aligned Speech Tokens
  4. 2024Speaking in Wavelet Domain: A Simple and Efficient Approach to Speed up Speech Diffusion Model
  5. 2022BDDM: Bilateral Denoising Diffusion Models for Fast and High-Quality Speech Synthesis
  6. 2022FastDiff: A Fast Conditional Diffusion Model for High-Quality Speech Synthesis
  7. 2022TransFusion: Transcribing Speech with Multinomial Diffusion
  8. 2026Fast Text-to-Audio Generation with One-Step Sampling via Energy-Scoring and Auxiliary Contextual Representation Distillation ↗