mlsanityData to 7 Oct 2026

DiffWave: A Versatile Diffusion Model for Audio Synthesis

arXiv 2009.09761 ↗First posted 21 Sep 2020eess.AS cs.CL cs.LGcited by 1,339

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

In this work, we propose DiffWave, a versatile diffusion probabilistic model for conditional and unconditional waveform generation. The model is non-autoregressive, and converts the white noise signal into structured waveform through a Markov chain with a constant number of steps at synthesis. It is efficiently trained by optimizing a variant of variational bound on the data likelihood. DiffWave produces high-fidelity audios in different waveform generation tasks, including neural vocoding conditioned on mel spectrogram, class-conditional generation, and unconditional generation. We demonstrate that DiffWave matches a strong WaveNet vocoder in terms of speech quality (MOS: 4.44 versus 4.43), while synthesizing orders of magnitude faster. In particular, it significantly outperforms autoregressive and GAN-based waveform models in the challenging unconditional generation task in terms of audio quality and sample diversity from various automatic and human evaluations.

Key facts. Cited by 1,339 papers in the map (data through 7 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 43 of its 56 references that are in the map. First posted on arXiv 21 Sep 2020.

Builds on

43 of 56 references.

References · 43

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  5. 2017Neural Discrete Representation Learningcited by 4,650
  6. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  7. 2013Auto-Encoding Variational Bayescited by 3,980
  8. 2016Improved Techniques for Training GANscited by 2,938
  9. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  10. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
  11. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  12. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,036
31 more
  1. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
  2. 2018Speech Commands: A Dataset for Limited-Vocabulary Speech Recognitioncited by 741
  3. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 620
  4. 2017Tacotron: Towards End-to-End Speech Synthesiscited by 487
  5. 2019MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesiscited by 351
  6. 2019Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogramcited by 337
  7. 2018WaveGlow: A Flow-based Generative Network for Speech Synthesiscited by 308
  8. 2018Adversarial Audio Synthesiscited by 222
  9. 2018Efficient Neural Audio Synthesiscited by 218
  10. 2017Parallel WaveNet: Fast High-Fidelity Speech Synthesiscited by 214
  11. 2020DDSP: Differentiable Digital Signal Processingcited by 177
  12. 2019GANSynth: Adversarial Neural Audio Synthesiscited by 114
  13. 2016SampleRNN: An Unconditional End-to-End Neural Audio Generation Modelcited by 110
  14. 2019High Fidelity Speech Synthesis with Adversarial Networkscited by 90
  15. 2017Deep Voice: Real-time Neural Text-to-Speechcited by 83
  16. 2017DeLiGAN : Generative Adversarial Networks for Diverse and Limited Datacited by 77
  17. 2020End-to-End Adversarial Text-to-Speechcited by 77
  18. 2017Deep Voice 2: Multi-Speaker Neural Text-to-Speechcited by 75
  19. 2018ClariNet: Parallel Wave Generation in End-to-End Text-to-Speechcited by 70
  20. 2018On GANs and GMMscited by 62
  21. 2018FloWaveNet : A Generative Flow for Raw Audiocited by 54
  22. 2020Flowtron: an Autoregressive Flow-based Generative Network for Text-to-Speech Synthesiscited by 54
  23. 2019WaveFlow: A Compact Flow-based Model for Raw Audiocited by 51
  24. 2017A Wavenet for Speech Denoisingcited by 47
  25. 2019MelNet: A Generative Model for Audio in the Frequency Domaincited by 41
  26. 2017Variational Walkback: Learning a Transition Operator as a Stochastic Recurrent Netcited by 30
  27. 2018Neural source-filter-based waveform model for statistical parametric speech synthesiscited by 21
  28. 2018End-to-end music source separation: is it possible in the waveform domain?cited by 19
  29. 2019A Neural Vocoder with Hierarchical Generation of Amplitude and Phase Spectra for Statistical Parametric Speech Synthesiscited by 10
  30. 2018Conditional WaveGANcited by 7
  31. 2020Conditional Spoken Digit Generation with StyleGANcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 66; 2022: 156; 2023: 355; 2024: 339; 2025: 278; 2026: 144.

1,339 papers.

Sort:

By area:

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  2. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  3. 2022Classifier-Free Diffusion Guidancecited by 5,717
  4. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  5. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  6. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  7. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  8. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  9. 2022Video Diffusion Modelscited by 2,280
  10. 2023Consistency Modelscited by 1,778
  11. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  12. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
38 more
  1. 2021Palette: Image-to-Image Diffusion Modelscited by 1,296
  2. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,155
  3. 2021Variational Diffusion Modelscited by 1,145
  4. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,107
  5. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  6. 2023One-step Diffusion with Distribution Matching Distillationcited by 978
  7. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
  8. 2022On Distillation of Guided Diffusion Modelscited by 721
  9. 2022Pseudo Numerical Methods for Diffusion Models on Manifoldscited by 695
  10. 2021Maximum Likelihood Training of Score-Based Diffusion Modelscited by 670
  11. 2023ModelScope Text-to-Video Technical Reportcited by 651
  12. 2021Score-based Generative Modeling in Latent Spacecited by 634
  13. 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Modelscited by 634
  14. 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 611
  15. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 606
  16. 2021Diffusion Schr"odinger Bridge with Applications to Score-Based Generative Modelingcited by 594
  17. 2023AudioLDM: Text-to-Audio Generation with Latent Diffusion Modelscited by 592
  18. 2021CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputationcited by 577
  19. 2022LION: Latent Point Diffusion Models for 3D Shape Generationcited by 552
  20. 2021Tackling the Generative Learning Trilemma with Denoising Diffusion GANscited by 515
  21. 2022All are Worth Words: A ViT Backbone for Diffusion Modelscited by 498
  22. 2021Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speechcited by 445
  23. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Modelscited by 439
  24. 2023DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Modelscited by 431
  25. 2023Universal Guidance for Diffusion Modelscited by 426
  26. 2021SRDiff: Single Image Super-Resolution with Diffusion Probabilistic Modelscited by 415
  27. 2023Simple diffusion: End-to-end diffusion for high resolution imagescited by 413
  28. 2023Analyzing and Improving the Training Dynamics of Diffusion Modelscited by 396
  29. 2023InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generationcited by 387
  30. 2022TabDDPM: Modelling Tabular Data with Diffusion Modelscited by 373
  31. 2024One Step Diffusion via Shortcut Modelscited by 373
  32. 2022PhysDiff: Physics-Guided Human Motion Diffusion Modelcited by 365
  33. 2023AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretrainingcited by 343
  34. 2021Autoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series Forecastingcited by 332
  35. 2022Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Modelscited by 331
  36. 2023Synthetic Data from Diffusion Models Improves ImageNet Classificationcited by 326
  37. 2021Score-based diffusion models for accelerated MRIcited by 308
  38. 2022Perception Prioritized Training of Diffusion Modelscited by 301

50 most cited of 1,339 All 1,339 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Audio Generation Through Score-Based Generative Modeling: Design Principles and Implementation
  2. 2026AudioX-Turbo: A Unified Framework for Efficient Anything-to-Audio Generation
  3. 2026DiTAR+: Dual Optimization for Robust Autoregressive Diffusion Speech Synthesis
  4. 2026TempoSyncDiff: Distilled Temporally-Consistent Diffusion for Low-Latency Audio-Driven Talking Head Generation
  5. 2025Post-Training Quantization for Audio Diffusion Transformers
  6. 2026FastWave: Optimized Diffusion Model for Audio Super-Resolution
  7. 2023ArchiSound: Audio Generation with Diffusion
  8. 2024Autoregressive Diffusion Transformer for Text-to-Speech Synthesis