mlsanityData to 7 Oct 2026

Universal Speech Enhancement with Score-based Diffusion

arXiv 2206.03065 ↗First posted 7 Jun 2022cs.SD cs.LG eess.AScited by 75

Not on the map. Why?

Cited by 45 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Removing background noise from speech audio has been the subject of considerable effort, especially in recent years due to the rise of virtual communication and amateur recordings. Yet background noise is not the only unpleasant disturbance that can prevent intelligibility: reverb, clipping, codec artifacts, problematic equalization, limited bandwidth, or inconsistent loudness are equally disturbing and ubiquitous. In this work, we propose to consider the task of speech enhancement as a holistic endeavor, and present a universal speech enhancement system that tackles 55 different distortions at the same time. Our approach consists of a generative model that employs score-based diffusion, together with a multi-resolution conditioning network that performs enhancement with mixture density networks. We show that this approach significantly outperforms the state of the art in a subjective test performed by expert listeners. We also show that it achieves competitive objective scores with just 4-8 diffusion steps, despite not considering any particular strategy for fast sampling. We hope that both our methodology and technical contributions encourage researchers and practitioners to adopt a universal approach to speech enhancement, possibly framing it as a generative task.

Key facts. Cited by 75 papers in the map (data through 7 Oct 2026). Builds on 46 of its 70 references that are in the map. First posted on arXiv 7 Jun 2022.

Builds on

46 of 70 references.

References · 46

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  5. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  6. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  7. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  8. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,036
  9. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 620
  10. 2021Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speechcited by 445
  11. 2019Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogramcited by 337
  12. 2020Real Time Speech Enhancement in the Waveform Domaincited by 183
34 more
  1. 2017SEGAN: Speech Enhancement Generative Adversarial Networkcited by 165
  2. 2021Diff-TTS: A Denoising Diffusion Model for Text-to-Speechcited by 156
  3. 2022Conditional Diffusion Probabilistic Model for Speech Enhancementcited by 132
  4. 2020The INTERSPEECH 2020 Deep Noise Suppression Challenge: Datasets, Subjective Testing Framework, and Challenge Resultscited by 128
  5. 2021MetricGAN+: An Improved Version of MetricGAN for Speech Enhancementcited by 114
  6. 2018CREPE: A Convolutional Representation for Pitch Estimationcited by 109
  7. 2022Speech Enhancement with Score-Based Generative Models in the Complex STFT Domaincited by 102
  8. 2020Adversarial score matching and improved sampling for image generationcited by 95
  9. 2020FullSubNet: A Full-Band and Sub-Band Fusion Model for Real-Time Single-Channel Speech Enhancementcited by 66
  10. 2021A Study on Speech Enhancement Based on Diffusion Probabilistic Modelcited by 52
  11. 2021NU-Wave: A Diffusion Probabilistic Model for Neural Audio Upsamplingcited by 50
  12. 2021VoiceFixer: Toward General Speech Restoration with Neural Vocodercited by 50
  13. 2017A Wavenet for Speech Denoisingcited by 47
  14. 2021CRASH: Raw Audio Score-based Generative Modeling for Controllable High-resolution Drum Sound Synthesiscited by 37
  15. 2020Upsampling artifacts in neural audio synthesiscited by 28
  16. 2020PoCoNet: Better Speech Enhancement with Frequency-Positional Embeddings, Semi-Supervised Conversational Data, and Biased Losscited by 23
  17. 2019Learning Sound Event Classifiers from Web Audio with Noisy Labelscited by 22
  18. 2020Improving GANs for Speech Enhancementcited by 22
  19. 2020Real-time Speech Frequency Bandwidth Extensioncited by 22
  20. 2021DCCRN+: Channel-wise Subband DCCRN with SNR Estimation for Speech Enhancementcited by 22
  21. 2021WSRGlow: A Glow-based Waveform Generative Model for Audio Super-Resolutioncited by 19
  22. 2021A Flow-Based Neural Network for Time Domain Speech Enhancementcited by 16
  23. 2020SESQA: semi-supervised learning for speech quality assessmentcited by 13
  24. 2020Interactive Speech and Noise Modeling for Speech Enhancementcited by 12
  25. 2021WARP-Q: Quality Prediction For Generative Neural Speech Codecscited by 12
  26. 2020Perceptual Loss based Speech Denoising with an ensemble of Audio Pattern Recognition and Self-Supervised Modelscited by 10
  27. 2021High Fidelity Speech Regeneration with Application to Speech Enhancementcited by 10
  28. 2021Restoring degraded speech via a modified diffusion modelcited by 10
  29. 2019Towards Generalized Speech Enhancement with Generative Adversarial Networkscited by 8
  30. 2019Parametric Resynthesis with neural vocoderscited by 8
  31. 2020Source coding of audio signals with a generative modelcited by 7
  32. 2019Speaker independence of neural vocoders and their effect on parametric resynthesis speech enhancementcited by 4
  33. 2021On tuning consistent annealed sampling for denoising score matchingcited by 4
  34. 2021A Two-stage Complex Network using Cycle-consistent Generative Adversarial Networks for Speech Enhancementcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 12; 2023: 17; 2024: 14; 2025: 23; 2026: 9.

75 papers.

Sort:

By area:

  1. 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scalecited by 373
  2. 2022Speech Enhancement and Dereverberation with Diffusion-based Generative Modelscited by 195
  3. 2023Text-to-Audio Generation using Instruction-Tuned LLM and Latent Diffusion Modelcited by 145
  4. 2022StoRM: A Diffusion-based Stochastic Regeneration Model for Speech Enhancement and Dereverberationcited by 105
  5. 2022Soft Diffusion: Score Matching for General Corruptionscited by 101
  6. 2022Blurring Diffusion Modelscited by 89
  7. 2023A Survey on Audio Diffusion Models: Text To Speech Synthesis and Enhancement in Generative AIcited by 68
  8. 2024A Survey on Diffusion Models for Time Series and Spatio-Temporal Datacited by 66
  9. 2023SpeechX: Neural Codec Language Model as a Versatile Speech Transformercited by 64
  10. 2023Multi-Source Diffusion Models for Simultaneous Music Generation and Separationcited by 63
  11. 2022Solving Audio Inverse Problems with a Diffusion Modelcited by 62
  12. 2022Diffusion-based Generative Speech Source Separationcited by 41
38 more
  1. 2022Analysing Diffusion-based Generative Approaches versus Discriminative Approaches for Speech Restorationcited by 34
  2. 2022Full-band General Audio Synthesis with Score-based Diffusioncited by 32
  3. 2022Cold Diffusion for Speech Enhancementcited by 31
  4. 2023Reducing the Prior Mismatch of Stochastic Differential Equations for Diffusion-based Speech Enhancementcited by 30
  5. 2024MaskSR: Masked Language Model for Full-band Speech Restorationcited by 24
  6. 2022CMGAN: Conformer-Based Metric-GAN for Monaural Speech Enhancementcited by 22
  7. 2023Miipher: A Robust Speech Restoration Model Integrating Self-Supervised Speech and Text Representationscited by 22
  8. 2023Blind Audio Bandwidth Extension: A Diffusion-Based Zero-Shot Approachcited by 21
  9. 2023The Ethical Implications of Generative Audio Models: A Systematic Literature Reviewcited by 18
  10. 2023Target Speech Extraction with Conditional Diffusion Modelcited by 15
  11. 2022Diffiner: A Versatile Diffusion-based Generative Refiner for Speech Enhancementcited by 14
  12. 2023Unsupervised speech enhancement with diffusion-based generative modelscited by 12
  13. 2025A2SB: Audio-to-Audio Schrodinger Bridgescited by 11
  14. 2023Diffusion-Based Speech Enhancement with Joint Generative and Predictive Decoderscited by 10
  15. 2023Noise-aware Speech Enhancement using Diffusion Probabilistic Modelcited by 9
  16. 2022ReVISE: Self-Supervised Speech Resynthesis with Visual Input for Universal and Generalized Speech Enhancementcited by 8
  17. 2024Diffusion-based Speech Enhancement with Schr"odinger Bridge and Symmetric Noise Schedulecited by 8
  18. 2024High-Resolution Speech Restoration with Latent Diffusion Modelcited by 8
  19. 2024Diffusion-based Unsupervised Audio-visual Speech Enhancementcited by 8
  20. 2023UnDiff: Unsupervised Voice Restoration with Unconditional Diffusion Modelcited by 7
  21. 2024Self-Supervised Speech Quality Estimation and Enhancement Using Only Clean Speechcited by 7
  22. 2024FINALLY: fast and universal speech enhancement with studio-like qualitycited by 7
  23. 2025Speech Enhancement Using Continuous Embeddings of Neural Audio Codeccited by 7
  24. 2025Universal Speech Enhancement with Regression and Generative Mambacited by 7
  25. 2024Speaking in Wavelet Domain: A Simple and Efficient Approach to Speed up Speech Diffusion Modelcited by 6
  26. 2025DiTSE: High-Fidelity Generative Speech Enhancement via Latent Diffusion Transformerscited by 6
  27. 2023AV2Wav: Diffusion-Based Re-synthesis from Continuous Self-supervised Features for Audio-Visual Speech Enhancementcited by 5
  28. 2024Vibravox: A Dataset of French Speech Captured with Body-conduction Audio Sensorscited by 5
  29. 2024Joint Semantic Knowledge Distillation and Masked Acoustic Modeling for Full-band Speech Restoration with Improved Intelligibilitycited by 5
  30. 2025MGE-LDM: Joint Latent Diffusion for Simultaneous Music Generation and Source Extractioncited by 4
  31. 2025A Composite Predictive-Generative Approach to Monaural Universal Speech Enhancementcited by 4
  32. 2025VoiceBridge: General Speech Restoration with One-step Latent Bridge Modelscited by 4
  33. 2025UniSE: A Unified Framework for Decoder-Only Autoregressive LM-Based Speech Enhancementcited by 4
  34. 2023Diffusion-based Signal Refiner for Speech Enhancement and Separationcited by 3
  35. 2025ProSE: Diffusion Priors for Speech Enhancementcited by 3
  36. 2025EDNet: A Versatile Speech Enhancement Framework with Gating Mamba Mechanism and Phase Shift-Invariant Trainingcited by 3
  37. 2024Pre-training Feature Guided Diffusion Model for Speech Enhancementcited by 2
  38. 2025TS-URGENet: A Three-stage Universal Robust and Generalizable Speech Enhancement Networkcited by 2

50 most cited of 75 All 75 (JSON)