# A Study on Speech Enhancement Based on Diffusion Probabilistic Model arXiv: https://arxiv.org/abs/2107.11876 ยท HTML: https://mlsanity.com/paper/2107.11876/ ## Key facts - Cited by 53 papers in the map (data through 14 Oct 2026). - Placed in Audio, speech and music, one of 14 directions in Diffusion models. - Builds on 19 of its 43 references that are in the map. - First posted on arXiv 25 Jul 2021. ## Placed in - Diffusion models / [Audio, speech and music](https://mlsanity.com/diffusion/audio-speech-and-music/) ## Builds on (most cited first, 19 of 19) - [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020) - [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015) - [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021) - [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016) - [DiffWave: A Versatile Diffusion Model for Audio Synthesis](https://mlsanity.com/paper/2009.09761/) (2020) - [Supervised Speech Separation Based on Deep Learning: An Overview](https://mlsanity.com/paper/1708.07524/) (2017) - [Real Time Speech Enhancement in the Waveform Domain](https://mlsanity.com/paper/2006.12847/) (2020) - [SEGAN: Speech Enhancement Generative Adversarial Network](https://mlsanity.com/paper/1703.09452/) (2017) - [MetricGAN: Generative Adversarial Networks based Black-box Metric Scores Optimization for Speech Enhancement](https://mlsanity.com/paper/1905.04874/) (2019) - [DiffSVC: A Diffusion Probabilistic Model for Singing Voice Conversion](https://mlsanity.com/paper/2105.13871/) (2021) - [A Recurrent Variational Autoencoder for Speech Enhancement](https://mlsanity.com/paper/1910.10942/) (2019) - [Speech Denoising with Deep Feature Losses](https://mlsanity.com/paper/1806.10522/) (2018) - [End-to-End Waveform Utterance Enhancement for Direct Evaluation Metrics Optimization by Fully Convolutional Neural Networks](https://mlsanity.com/paper/1709.03658/) (2017) - [Improving GANs for Speech Enhancement](https://mlsanity.com/paper/2001.05532/) (2020) - [Conditional Generative Adversarial Networks for Speech Enhancement and Noise-Robust Speaker Verification](https://mlsanity.com/paper/1709.01703/) (2017) - [Characterizing Speech Adversarial Examples Using Self-Attention U-Net Enhancement](https://mlsanity.com/paper/2003.13917/) (2020) - [A Flow-Based Neural Network for Time Domain Speech Enhancement](https://mlsanity.com/paper/2106.09008/) (2021) - [Exploring Deep Hybrid Tensor-to-Vector Network Architectures for Regression Based Speech Enhancement](https://mlsanity.com/paper/2007.13024/) (2020) - [Student-Teacher Learning for BLSTM Mask-based Speech Enhancement](https://mlsanity.com/paper/1803.10013/) (2018) ## Built on by (most cited first, 30 of 53) - [Speech Enhancement and Dereverberation with Diffusion-based Generative Models](https://mlsanity.com/paper/2208.05830/) (2022) - [Text-to-Audio Generation using Instruction-Tuned LLM and Latent Diffusion Model](https://mlsanity.com/paper/2304.13731/) (2023) - [Conditional Diffusion Probabilistic Model for Speech Enhancement](https://mlsanity.com/paper/2202.05256/) (2022) - [Speech Enhancement with Score-Based Generative Models in the Complex STFT Domain](https://mlsanity.com/paper/2203.17004/) (2022) - [Universal Speech Enhancement with Score-based Diffusion](https://mlsanity.com/paper/2206.03065/) (2022) - [An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimization](https://mlsanity.com/paper/2404.07771/) (2024) - [A Survey on Audio Diffusion Models: Text To Speech Synthesis and Enhancement in Generative AI](https://mlsanity.com/paper/2303.13336/) (2023) - [Multi-Source Diffusion Models for Simultaneous Music Generation and Separation](https://mlsanity.com/paper/2302.02257/) (2023) - [On the Design Fundamentals of Diffusion Models: A Survey](https://mlsanity.com/paper/2306.04542/) (2023) - [Generative Pre-training for Speech with Flow Matching](https://mlsanity.com/paper/2310.16338/) (2023) - [BinauralGrad: A Two-Stage Conditional Diffusion Probabilistic Model for Binaural Audio Synthesis](https://mlsanity.com/paper/2205.14807/) (2022) - [Diffusion-based Generative Speech Source Separation](https://mlsanity.com/paper/2210.17327/) (2022) - [Analysing Diffusion-based Generative Approaches versus Discriminative Approaches for Speech Restoration](https://mlsanity.com/paper/2211.02397/) (2022) - [Diffusion Models for Audio Restoration](https://mlsanity.com/paper/2402.09821/) (2024) - [Cold Diffusion for Speech Enhancement](https://mlsanity.com/paper/2211.02527/) (2022) - [Reducing the Prior Mismatch of Stochastic Differential Equations for Diffusion-based Speech Enhancement](https://mlsanity.com/paper/2302.14748/) (2023) - [Conditioning and Sampling in Variational Diffusion Models for Speech Super-Resolution](https://mlsanity.com/paper/2210.15793/) (2022) - [The Ethical Implications of Generative Audio Models: A Systematic Literature Review](https://mlsanity.com/paper/2307.05527/) (2023) - [Diffiner: A Versatile Diffusion-based Generative Refiner for Speech Enhancement](https://mlsanity.com/paper/2210.17287/) (2022) - [Single and Few-step Diffusion for Generative Speech Enhancement](https://mlsanity.com/paper/2309.09677/) (2023) - [SRTNet: Time Domain Speech Enhancement Via Stochastic Refinement](https://mlsanity.com/paper/2210.16805/) (2022) - [Metric-oriented Speech Enhancement using Diffusion Probabilistic Model](https://mlsanity.com/paper/2302.11989/) (2023) - [Distilling ODE Solvers of Diffusion Models into Smaller Steps](https://mlsanity.com/paper/2309.16421/) (2023) - [Investigating the Design Space of Diffusion Models for Speech Enhancement](https://mlsanity.com/paper/2312.04370/) (2023) - [Noise-aware Speech Enhancement using Diffusion Probabilistic Model](https://mlsanity.com/paper/2307.08029/) (2023) - [DiffPhase: Generative Diffusion-based STFT Phase Retrieval](https://mlsanity.com/paper/2211.04332/) (2022) - [Diffusion-Based Speech Enhancement in Matched and Mismatched Conditions Using a Heun-Based Sampler](https://mlsanity.com/paper/2312.02683/) (2023) - [Diffusion-based Speech Enhancement with Schr"odinger Bridge and Symmetric Noise Schedule](https://mlsanity.com/paper/2409.05116/) (2024) - [An Analysis of the Variance of Diffusion-based Speech Enhancement](https://mlsanity.com/paper/2402.00811/) (2024) - [SDEMG: Score-based Diffusion Model for Surface Electromyographic Signal Denoising](https://mlsanity.com/paper/2402.03808/) (2024) ## Abstract (arXiv) Diffusion probabilistic models have demonstrated an outstanding capability to model natural images and raw audio waveforms through a paired diffusion and reverse processes. The unique property of the reverse process (namely, eliminating non-target signals from the Gaussian noise and noisy signals) could be utilized to restore clean signals. Based on this property, we propose a diffusion probabilistic model-based speech enhancement (DiffuSE) model that aims to recover clean speech signals from noisy signals. The fundamental architecture of the proposed DiffuSE model is similar to that of DiffWave--a high-quality audio waveform generation model that has a relatively low computational cost and footprint. To attain better enhancement performance, we designed an advanced reverse process, termed the supportive reverse process, which adds noisy speech in each time-step to the predicted speech. The experimental results show that DiffuSE yields performance that is comparable to related audio generative models on the standardized Voice Bank corpus SE task. Moreover, relative to the generally suggested full sampling schedule, the proposed supportive reverse process especially improved the fast sampling, taking few steps to yield better enhancement results over the conventional full step inference process.