mlsanityData to 7 Oct 2026

FastDiff: A Fast Conditional Diffusion Model for High-Quality Speech Synthesis

arXiv 2204.09934 ↗First posted 21 Apr 2022eess.AS cs.LG cs.SDcited by 136

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

Denoising diffusion probabilistic models (DDPMs) have recently achieved leading performances in many generative tasks. However, the inherited iterative sampling process costs hindered their applications to speech synthesis. This paper proposes FastDiff, a fast conditional diffusion model for high-quality speech synthesis. FastDiff employs a stack of time-aware location-variable convolutions of diverse receptive field patterns to efficiently model long-term time dependencies with adaptive conditions. A noise schedule predictor is also adopted to reduce the sampling steps without sacrificing the generation quality. Based on FastDiff, we design an end-to-end text-to-speech synthesizer, FastDiff-TTS, which generates high-fidelity speech waveforms without any intermediate feature (e.g., Mel-spectrogram). Our evaluation of FastDiff demonstrates the state-of-the-art results with higher-quality (MOS 4.28) speech samples. Also, FastDiff enables a sampling speed of 58x faster than real-time on a V100 GPU, making diffusion models practically applicable to speech synthesis deployment for the first time. We further show that FastDiff generalized well to the mel-spectrogram inversion of unseen speakers, and FastDiff-TTS outperformed other competing methods in end-to-end text-to-speech synthesis. Audio samples are available at urlhttps://FastDiff.github.io/.

Key facts. Cited by 136 papers in the map (data through 7 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 25 of its 31 references that are in the map. First posted on arXiv 21 Apr 2022.

Builds on

25 of 31 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022BDDM: Bilateral Denoising Diffusion Models for Fast and High-Quality Speech Synthesis25 Mar
  2. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesis21 Sep

Other references · 23

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2020Denoising Diffusion Implicit Modelscited by 9,655
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  6. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  7. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  8. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
  9. 2022Pseudo Numerical Methods for Diffusion Models on Manifoldscited by 695
  10. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 620
  11. 2017Generative Adversarial Networks: An Overviewcited by 588
  12. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 555
11 more
  1. 2021Tackling the Generative Learning Trilemma with Denoising Diffusion GANscited by 515
  2. 2018WaveGlow: A Flow-based Generative Network for Speech Synthesiscited by 308
  3. 2020Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Searchcited by 269
  4. 2021DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanismcited by 154
  5. 2021Noise Estimation for Generative Diffusion Modelscited by 94
  6. 2020End-to-End Adversarial Text-to-Speechcited by 77
  7. 2021UnivNet: A Neural Vocoder with Multi-Resolution Spectrogram Discriminators for High-Fidelity Waveform Generationcited by 71
  8. 2021WaveGrad 2: Iterative Refinement for Text-to-Speech Synthesiscited by 65
  9. 2021Multi-Singer: Fast Multi-Singer Singing Voice Vocoder With A Large-Scale Corpuscited by 60
  10. 2021PortaSpeech: Portable and High-Quality Generative Text-to-Speechcited by 47
  11. 2021LVCNet: Efficient Condition-Dependent Modeling Network for Waveform Generationcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 12; 2023: 59; 2024: 33; 2025: 21; 2026: 11.

136 papers.

Sort:

By area:

  1. 2023Universal Guidance for Diffusion Modelscited by 426
  2. 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scalecited by 373
  3. 2023Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Modelscited by 323
  4. 2023AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Headcited by 264
  5. 2023StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Modelscited by 148
  6. 2022ProDiff: Progressive Fast Diffusion Model For High-Quality Text-to-Speechcited by 147
  7. 2023Text-to-Audio Generation using Instruction-Tuned LLM and Latent Diffusion Modelcited by 145
  8. 2023Editing Implicit Assumptions in Text-to-Image Diffusion Modelscited by 128
  9. 2023Diffusion Models for Image Restoration and Enhancement: A Comprehensive Surveycited by 108
  10. 2022How to Backdoor Diffusion Models?cited by 102
  11. 2023A Survey on Audio Diffusion Models: Text To Speech Synthesis and Enhancement in Generative AIcited by 68
  12. 2023StyleDiffusion: Prompt-Embedding Inversion for Text-Based Editingcited by 68
38 more
  1. 2023VillanDiffusion: A Unified Backdoor Attack Framework for Diffusion Modelscited by 68
  2. 2024A Survey on Diffusion Models for Time Series and Spatio-Temporal Datacited by 66
  3. 2023One-Step Diffusion Distillation via Deep Equilibrium Modelscited by 60
  4. 2023FaceTalk: Audio-Driven Motion Diffusion for Neural Parametric Head Modelscited by 55
  5. 2022Unsupervised Representation Learning from Pre-trained Diffusion Probabilistic Modelscited by 53
  6. 2023Improving 3D Imaging with Pre-Trained Perpendicular 2D Diffusion Modelscited by 53
  7. 2023An Efficient Membership Inference Attack for the Diffusion Model by Proximal Initializationcited by 53
  8. 2023Mega-TTS: Zero-Shot Text-to-Speech at Scale with Intrinsic Inductive Biascited by 51
  9. 2023Efficient Neural Music Generationcited by 50
  10. 2023VoiceFlow: Efficient Text-to-Speech with Rectified Flow Matchingcited by 49
  11. 2023TransFusion -- A Transparency-Based Diffusion Model for Anomaly Detectioncited by 43
  12. 2023DDDM-VC: Decoupled Denoising Diffusion Models with Disentangled Representation and Prior Mixup for Verified Robust Voice Conversioncited by 38
  13. 2024HIR-Diff: Unsupervised Hyperspectral Image Restoration Via Improved Diffusion Modelscited by 38
  14. 2022EmoDiff: Intensity Controllable Emotional Text-to-Speech with Soft-Label Guidancecited by 37
  15. 2024A Good Score Does not Lead to A Good Generative Modelcited by 36
  16. 2022TranSpeech: Speech-to-Speech Translation With Bilateral Perturbationcited by 34
  17. 2023Multi-Architecture Multi-Expert Diffusion Modelscited by 34
  18. 2023Diff-HierVC: Diffusion-based Hierarchical Voice Conversion with Robust Pitch Generation and Masked Prior for Zero-shot Speaker Adaptationcited by 30
  19. 2023Degeneration-Tuning: Using Scrambled Grid shield Unwanted Concepts from Stable Diffusioncited by 27
  20. 2023Intelligent Model Update Strategy for Sequential Recommendationcited by 25
  21. 2023ViT-TTS: Visual Text-to-Speech with Scalable Diffusion Transformercited by 24
  22. 2023DiffDance: Cascaded Human Motion Diffusion Model for Dance Generationcited by 23
  23. 2023ConsistencyTTA: Accelerating Diffusion-Based Text-to-Audio Generation with Consistency Distillationcited by 22
  24. 2024Towards Controllable Speech Synthesis in the Era of Large Language Models: A Systematic Surveycited by 22
  25. 2022GenerSpeech: Towards Style Transfer for Generalizable Out-Of-Domain Text-to-Speechcited by 21
  26. 2022ResGrad: Residual Denoising Diffusion Probabilistic Models for Text to Speechcited by 21
  27. 2023CoMusion: Towards Consistent Stochastic Human Motion Prediction via Motion Diffusioncited by 21
  28. 2022Controllable Data Generation by Deep Learning: A Reviewcited by 20
  29. 2023Delving into the Adversarial Robustness of Federated Learningcited by 19
  30. 2023Generative Spoken Language Model based on continuous word-sized audio tokenscited by 19
  31. 2024MeLFusion: Synthesizing Music from Image and Language Cues using Diffusion Modelscited by 19
  32. 2022DriftRec: Adapting diffusion models to blind JPEG restorationcited by 15
  33. 2023The Age of Synthetic Realities: Challenges and Opportunitiescited by 15
  34. 2024CoVoMix: Advancing Zero-Shot Speech Generation for Human-like Multi-talker Conversationscited by 15
  35. 2023ShiftDDPMs: Exploring Conditional Diffusion Models by Shifting Diffusion Trajectoriescited by 14
  36. 2023MixSpeech: Cross-Modality Self-Learning with Audio-Visual Stream Mixup for Visual Speech Translation and Recognitioncited by 14
  37. 2024Towards Black-Box Membership Inference Attack for Diffusion Modelscited by 14
  38. 2023RMSSinger: Realistic-Music-Score based Singing Voice Synthesiscited by 12

50 most cited of 136 All 136 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2021Diff-TTS: A Denoising Diffusion Model for Text-to-Speech
  2. 2022DiffGAN-TTS: High-Fidelity and Efficient Text-to-Speech with Denoising Diffusion GANs
  3. 2024DMOSpeech: Direct Metric Optimization via Distilled Diffusion Model in Zero-Shot Speech Synthesis
  4. 2023CoMoSpeech: One-Step Speech and Singing Voice Synthesis via Consistency Model
  5. 2021Diffusion-Based Voice Conversion with Fast Maximum Likelihood Sampling Scheme
  6. 2021PriorGrad: Improving Conditional Denoising Diffusion Models with Data-Dependent Adaptive Prior
  7. 2023ReFlow-TTS: A Rectified Flow Model for High-fidelity Text-to-Speech
  8. 2026DiTAR+: Dual Optimization for Robust Autoregressive Diffusion Speech Synthesis