mlsanityData to 7 Oct 2026

BDDM: Bilateral Denoising Diffusion Models for Fast and High-Quality Speech Synthesis

arXiv 2203.13508 ↗First posted 25 Mar 2022eess.AS cs.AI cs.LGcited by 64

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

Diffusion probabilistic models (DPMs) and their extensions have emerged as competitive generative models yet confront challenges of efficient sampling. We propose a new bilateral denoising diffusion model (BDDM) that parameterizes both the forward and reverse processes with a schedule network and a score network, which can train with a novel bilateral modeling objective. We show that the new surrogate objective can achieve a lower bound of the log marginal likelihood tighter than a conventional surrogate. We also find that BDDM allows inheriting pre-trained score network parameters from any DPMs and consequently enables speedy and stable learning of the schedule network and optimization of a noise schedule for sampling. Our experiments demonstrate that BDDMs can generate high-fidelity audio samples with as few as three sampling steps. Moreover, compared to other state-of-the-art diffusion-based neural vocoders, BDDMs produce comparable or higher quality samples indistinguishable from human speech, notably with only seven sampling steps (143x faster than WaveGrad and 28.6x faster than DiffWave). We release our code at https://github.com/tencent-ailab/bddm.

Key facts. Cited by 64 papers in the map (data through 7 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 34 of its 47 references that are in the map. First posted on arXiv 25 Mar 2022.

Builds on

34 of 47 references.

Closest ancestors · 1

Same direction, newest first

  1. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesis21 Sep

Other references · 33

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2020Denoising Diffusion Implicit Modelscited by 9,655
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  6. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  7. 2022Generative Adversarial Networkscited by 5,603
  8. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  9. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  10. 2018Neural Ordinary Differential Equationscited by 3,733
  11. 2016Density estimation using Real NVPcited by 2,029
  12. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
21 more
  1. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
  2. 2019Normalizing Flows for Probabilistic Modeling and Inferencecited by 1,315
  3. 2021Variational Diffusion Modelscited by 1,145
  4. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  5. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,036
  6. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 620
  7. 2019Sliced Score Matching: A Scalable Approach to Density and Score Estimationcited by 381
  8. 2019MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesiscited by 351
  9. 2019Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogramcited by 337
  10. 2018WaveGlow: A Flow-based Generative Network for Speech Synthesiscited by 308
  11. 2019Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Designcited by 258
  12. 2021Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Modelscited by 251
  13. 2018Efficient Neural Audio Synthesiscited by 218
  14. 2017Parallel WaveNet: Fast High-Fidelity Speech Synthesiscited by 214
  15. 2021On Fast Sampling of Diffusion Probabilistic Modelscited by 170
  16. 2021Learning to Efficiently Sample from Diffusion Probabilistic Modelscited by 129
  17. 2019BIVA: A Very Deep Hierarchy of Latent Variables for Generative Modelingcited by 109
  18. 2021Noise Estimation for Generative Diffusion Modelscited by 94
  19. 2019High Fidelity Speech Synthesis with Adversarial Networkscited by 90
  20. 2020Multi-band MelGAN: Faster Waveform Generation for High-Quality Text-to-Speechcited by 63
  21. 2021Effective Low-Cost Time-Domain Audio Separation Using Globally Attentive Locally Recurrent Networkscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 11; 2023: 27; 2024: 16; 2025: 8; 2026: 2.

64 papers.

Sort:

By area:

  1. 2022Listen, Denoise, Action! Audio-Driven Motion Synthesis with Diffusion Modelscited by 188
  2. 2023PTQD: Accurate Post-Training Quantization for Diffusion Modelscited by 167
  3. 2022ProDiff: Progressive Fast Diffusion Model For High-Quality Text-to-Speechcited by 147
  4. 2022FastDiff: A Fast Conditional Diffusion Model for High-Quality Speech Synthesiscited by 136
  5. 2022StoRM: A Diffusion-based Stochastic Regeneration Model for Speech Enhancement and Dereverberationcited by 105
  6. 2023A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need?cited by 84
  7. 2023Diffusion Action Segmentationcited by 74
  8. 2023SA-Solver: Stochastic Adams Solver for Fast Sampling of Diffusion Modelscited by 71
  9. 2022Subspace Diffusion Generative Modelscited by 70
  10. 2023A Survey on Audio Diffusion Models: Text To Speech Synthesis and Enhancement in Generative AIcited by 68
  11. 2024Accelerating Diffusion Sampling with Optimized Time Stepscited by 67
  12. 2023FaceTalk: Audio-Driven Motion Diffusion for Neural Parametric Head Modelscited by 55
38 more
  1. 2022Score-based Generative Modeling Secretly Minimizes the Wasserstein Distancecited by 50
  2. 2023DDM2: Self-Supervised Diffusion MRI Denoising with Generative Diffusion Modelscited by 50
  3. 2023Efficient Neural Music Generationcited by 50
  4. 2023VoiceFlow: Efficient Text-to-Speech with Rectified Flow Matchingcited by 49
  5. 2023Exact Diffusion Inversion via Bi-directional Integration Approximationcited by 43
  6. 2024Diffusion Facial Forgery Detectioncited by 41
  7. 2022SpecGrad: Diffusion Probabilistic Model based Neural Vocoder with Adaptive Noise Spectral Shapingcited by 37
  8. 2022EmoDiff: Intensity Controllable Emotional Text-to-Speech with Soft-Label Guidancecited by 37
  9. 2024Foundation Models for Music: A Surveycited by 34
  10. 2024Diffusion Models for Audio Restorationcited by 32
  11. 2023Generating Visual Scenes from Touchcited by 26
  12. 2023Conditional Denoising Diffusion for Sequential Recommendationcited by 23
  13. 2023Distribution Shift Inversion for Out-of-Distribution Predictioncited by 23
  14. 2023Promptable Game Models: Text-Guided Game Simulation via Masked Diffusion Modelscited by 22
  15. 2022WaveFit: An Iterative and Non-autoregressive Neural Vocoder based on Fixed-Point Iterationcited by 20
  16. 2024DFADD: The Diffusion and Flow-Matching Based Audio Deepfake Datasetcited by 20
  17. 2024MeLFusion: Synthesizing Music from Image and Language Cues using Diffusion Modelscited by 19
  18. 2024GROOT: Generating Robust Watermark for Diffusion-Model-Based Audio Synthesiscited by 17
  19. 2022On Investigating the Conservative Property of Score-Based Generative Modelscited by 16
  20. 2023ChiroDiff: Modelling chirographic data with Diffusion Modelscited by 14
  21. 2023ReDi: Efficient Learning-Free Diffusion Inference via Trajectory Retrievalcited by 11
  22. 2023Lookahead Diffusion Probabilistic Models for Refining Mean Estimationcited by 11
  23. 2023Efficient Integrators for Diffusion Generative Modelscited by 11
  24. 2024FreGrad: Lightweight and Fast Frequency-aware Diffusion Vocodercited by 11
  25. 2023ERA-Solver: Error-Robust Adams Solver for Fast Sampling of Diffusion Probabilistic Modelscited by 9
  26. 2023Image Synthesis under Limited Data: A Survey and Taxonomycited by 9
  27. 2023DiffAnt: Diffusion Models for Action Anticipationcited by 9
  28. 2024AVI-Talking: Learning Audio-Visual Instructions for Expressive 3D Talking Face Generationcited by 8
  29. 2024Diffuse or Confuse: A Diffusion Deepfake Speech Datasetcited by 8
  30. 2023On Accelerating Diffusion-Based Sampling Process via Improved Integration Approximationcited by 7
  31. 2024Counterfactual Data Augmentation with Denoising Diffusion for Graph Anomaly Detectioncited by 7
  32. 2025WaveFM: A High-Fidelity and Efficient Vocoder Based on Flow Matchingcited by 7
  33. 2024TCAQ-DM: Timestep-Channel Adaptive Quantization for Diffusion Modelscited by 6
  34. 2022Diffusion Denoising Process for Perceptron Bias in Out-of-distribution Detectioncited by 5
  35. 2024Temporal Feature Matters: A Framework for Diffusion Model Quantizationcited by 5
  36. 2023Modeling Molecular Structures with Intrinsic Diffusion Modelscited by 4
  37. 2023Boosting Fast and High-Quality Speech Synthesis with Linear Diffusioncited by 4
  38. 2023Diverse and Expressive Speech Prosody Prediction with Denoising Diffusion Probabilistic Modelcited by 3

50 most cited of 64 All 64 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2021Bilateral Denoising Diffusion Models
  2. 2021PriorGrad: Improving Conditional Denoising Diffusion Models with Data-Dependent Adaptive Prior
  3. 2024Speaking in Wavelet Domain: A Simple and Efficient Approach to Speed up Speech Diffusion Model
  4. 2022Conditioning and Sampling in Variational Diffusion Models for Speech Super-Resolution
  5. 2023Separate And Diffuse: Using a Pretrained Diffusion Model for Improving Source Separation
  6. 2022InferGrad: Improving Diffusion Models for Vocoder by Considering Inference in Training
  7. 2023DiffAR: Denoising Diffusion Autoregressive Model for Raw Speech Waveform Generation
  8. 2024On the Semantic Latent Space of Diffusion-Based Text-to-Speech Models