# Diffusion Models for Audio Restoration

arXiv: https://arxiv.org/abs/2402.09821 · HTML: https://mlsanity.com/paper/2402.09821/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Placed in Audio, speech and music, one of 14 directions in Diffusion models.
- Builds on 30 of its 42 references that are in the map.
- First posted on arXiv 15 Feb 2024.

## Placed in

- Diffusion models / [Audio, speech and music](https://mlsanity.com/diffusion/audio-speech-and-music/)

## Builds on (most cited first, 30 of 30)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Diffusion Posterior Sampling for General Noisy Inverse Problems](https://mlsanity.com/paper/2209.14687/) (2022)
- [DiffWave: A Versatile Diffusion Model for Audio Synthesis](https://mlsanity.com/paper/2009.09761/) (2020)
- [WaveGrad: Estimating Gradients for Waveform Generation](https://mlsanity.com/paper/2009.00713/) (2020)
- [AudioLDM: Text-to-Audio Generation with Latent Diffusion Models](https://mlsanity.com/paper/2301.12503/) (2023)
- [Supervised Speech Separation Based on Deep Learning: An Overview](https://mlsanity.com/paper/1708.07524/) (2017)
- [Speech Enhancement and Dereverberation with Diffusion-based Generative Models](https://mlsanity.com/paper/2208.05830/) (2022)
- [ProDiff: Progressive Fast Diffusion Model For High-Quality Text-to-Speech](https://mlsanity.com/paper/2207.06389/) (2022)
- [Conditional Diffusion Probabilistic Model for Speech Enhancement](https://mlsanity.com/paper/2202.05256/) (2022)
- [StoRM: A Diffusion-based Stochastic Regeneration Model for Speech Enhancement and Dereverberation](https://mlsanity.com/paper/2212.11851/) (2022)
- [Speech Enhancement with Score-Based Generative Models in the Complex STFT Domain](https://mlsanity.com/paper/2203.17004/) (2022)
- [AUDIT: Audio Editing by Following Instructions with Latent Diffusion Models](https://mlsanity.com/paper/2304.00830/) (2023)
- [BDDM: Bilateral Denoising Diffusion Models for Fast and High-Quality Speech Synthesis](https://mlsanity.com/paper/2203.13508/) (2022)
- [Solving Audio Inverse Problems with a Diffusion Model](https://mlsanity.com/paper/2210.15228/) (2022)
- [Generative Pre-training for Speech with Flow Matching](https://mlsanity.com/paper/2310.16338/) (2023)
- [A Study on Speech Enhancement Based on Diffusion Probabilistic Model](https://mlsanity.com/paper/2107.11876/) (2021)
- [Diffusion-based Generative Speech Source Separation](https://mlsanity.com/paper/2210.17327/) (2022)
- [Reducing the Prior Mismatch of Stochastic Differential Equations for Diffusion-based Speech Enhancement](https://mlsanity.com/paper/2302.14748/) (2023)
- [Denoising Diffusion Gamma Models](https://mlsanity.com/paper/2110.05948/) (2021)
- [Blind Audio Bandwidth Extension: A Diffusion-Based Zero-Shot Approach](https://mlsanity.com/paper/2306.01433/) (2023)
- [Single and Few-step Diffusion for Generative Speech Enhancement](https://mlsanity.com/paper/2309.09677/) (2023)
- [Diffusion-Based Audio Inpainting](https://mlsanity.com/paper/2305.15266/) (2023)
- [Audio-Visual Speech Enhancement with Score-Based Generative Models](https://mlsanity.com/paper/2306.01432/) (2023)
- [Diffusion Posterior Sampling for Informed Single-Channel Dereverberation](https://mlsanity.com/paper/2306.12286/) (2023)
- [DiffPhase: Generative Diffusion-based STFT Phase Retrieval](https://mlsanity.com/paper/2211.04332/) (2022)

## Built on by (most cited first, 30 of 32)

- [TFG: Unified Training-Free Guidance for Diffusion Models](https://mlsanity.com/paper/2409.15761/) (2024)
- [Can Diffusion Models Learn Hidden Inter-Feature Rules Behind Images?](https://mlsanity.com/paper/2502.04725/) (2025)
- [Unsupervised Blind Joint Dereverberation and Room Acoustics Estimation with Diffusion Models](https://mlsanity.com/paper/2408.07472/) (2024)
- [Source Separation by Flow Matching](https://mlsanity.com/paper/2505.16119/) (2025)
- [SonicMaster: Towards Controllable All-in-One Music Restoration and Mastering](https://mlsanity.com/paper/2508.03448/) (2025)
- [Unsupervised Estimation of Nonlinear Audio Effects: Comparing Diffusion-Based and Adversarial approaches](https://mlsanity.com/paper/2504.04751/) (2025)
- [Schr"odinger Bridge Consistency Trajectory Models for Speech Enhancement](https://mlsanity.com/paper/2507.11925/) (2025)
- [3DiFACE: Synthesizing and Editing Holistic 3D Facial Animation](https://mlsanity.com/paper/2509.26233/) (2025)
- [A Survey of Deep Learning for Complex Speech Spectrograms](https://mlsanity.com/paper/2505.08694/) (2025)
- [Incomplete Data, Complete Dynamics: A Diffusion Approach](https://mlsanity.com/paper/2509.20098/) (2025)
- [Responsible Diffusion: A Comprehensive Survey on Safety, Ethics, and Trust in Diffusion Models](https://mlsanity.com/paper/2509.22723/) (2025)
- [LipDiffuser: Lip-to-Speech Generation with Conditional Diffusion Models](https://mlsanity.com/paper/2505.11391/) (2025)
- [A Semantic Information-based Hierarchical Speech Enhancement Method Using Factorized Codec and Diffusion Model](https://mlsanity.com/paper/2505.13843/) (2025)
- [MambAttention: Mamba with Multi-Head Attention for Generalizable Single-Channel Speech Enhancement](https://mlsanity.com/paper/2507.00966/) (2025)
- [User-guided Generative Source Separation](https://mlsanity.com/paper/2507.01339/) (2025)
- [Sealing The Backdoor: Unlearning Adversarial Text Triggers In Diffusion Models Using Knowledge Distillation](https://mlsanity.com/paper/2508.18235/) (2025)
- [ArtiFree: Detecting and Reducing Generative Artifacts in Diffusion-based Speech Enhancement](https://mlsanity.com/paper/2509.19495/) (2025)
- [Audio Decoding by Inverse Problem Solving](https://mlsanity.com/paper/2409.07858/) (2024)
- [LoyalDiffusion: A Diffusion Model Guarding Against Data Replication](https://mlsanity.com/paper/2412.01118/) (2024)
- [Miipher-2: A Universal Speech Restoration Model for Million-Hour Scale Data Restoration](https://mlsanity.com/paper/2505.04457/) (2025)
- [Audio Generation Through Score-Based Generative Modeling: Design Principles and Implementation](https://mlsanity.com/paper/2506.08457/) (2025)
- [Improving Inference-Time Optimisation for Vocal Effects Style Transfer with a Gaussian Prior](https://mlsanity.com/paper/2505.11315/) (2025)
- [Posterior Transition Modeling for Unsupervised Diffusion-Based Speech Enhancement](https://mlsanity.com/paper/2507.02391/) (2025)
- [Similarity-Guided Diffusion for Long-Gap Music Inpainting](https://mlsanity.com/paper/2509.16342/) (2025)
- [ADNAC: Audio Denoiser using Neural Audio Codec](https://mlsanity.com/paper/2511.01773/) (2025)
- [Single-step Controllable Music Bandwidth Extension With Flow Matching](https://mlsanity.com/paper/2601.14356/) (2026)
- [Beyond Accuracy: Evaluating Posterior Fidelity of Diffusion Inverse Solvers](https://mlsanity.com/paper/2602.04189/) (2026)
- [Absorbing Discrete Diffusion for Speech Enhancement](https://mlsanity.com/paper/2602.22417/) (2026)
- [Predictive-Generative Drift Decomposition for Speech Enhancement and Separation](https://mlsanity.com/paper/2605.06189/) (2026)
- [QDSB: Quantized Diffusion Schr"odinger Bridges](https://mlsanity.com/paper/2605.11983/) (2026)

## Abstract (arXiv)

With the development of audio playback devices and fast data transmission, the demand for high sound quality is rising for both entertainment and communications. In this quest for better sound quality, challenges emerge from distortions and interferences originating at the recording side or caused by an imperfect transmission pipeline. To address this problem, audio restoration methods aim to recover clean sound signals from the corrupted input data. We present here audio restoration algorithms based on diffusion models, with a focus on speech enhancement and music restoration tasks. Traditional approaches, often grounded in handcrafted rules and statistical heuristics, have shaped our understanding of audio signals. In the past decades, there has been a notable shift towards data-driven methods that exploit the modeling capabilities of DNNs. Deep generative models, and among them diffusion models, have emerged as powerful techniques for learning complex data distributions. However, relying solely on DNN-based learning approaches carries the risk of reducing interpretability, particularly when employing end-to-end models. Nonetheless, data-driven approaches allow more flexibility in comparison to statistical model-based frameworks, whose performance depends on distributional and statistical assumptions that can be difficult to guarantee. Here, we aim to show that diffusion models can combine the best of both worlds and offer the opportunity to design audio restoration algorithms with a good degree of interpretability and a remarkable performance in terms of sound quality. We explain the diffusion formalism and its application to the conditional generation of clean audio signals. We believe that diffusion models open an exciting field of research with the potential to spawn new audio restoration algorithms that are natural-sounding and remain robust in difficult acoustic situations.