# UnDiff: Unsupervised Voice Restoration with Unconditional Diffusion Model

arXiv: https://arxiv.org/abs/2306.00721 · HTML: https://mlsanity.com/paper/2306.00721/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Audio, speech and music, one of 14 directions in Diffusion models.
- Builds on 29 of its 36 references that are in the map.
- First posted on arXiv 1 Jun 2023.

## Placed in

- Diffusion models / [Audio, speech and music](https://mlsanity.com/diffusion/audio-speech-and-music/)

## Builds on (most cited first, 29 of 29)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [Diffusion Posterior Sampling for General Noisy Inverse Problems](https://mlsanity.com/paper/2209.14687/) (2022)
- [DiffWave: A Versatile Diffusion Model for Audio Synthesis](https://mlsanity.com/paper/2009.09761/) (2020)
- [HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis](https://mlsanity.com/paper/2010.05646/) (2020)
- [ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2108.02938/) (2021)
- [AudioLM: a Language Modeling Approach to Audio Generation](https://mlsanity.com/paper/2209.03143/) (2022)
- [AudioLDM: Text-to-Audio Generation with Latent Diffusion Models](https://mlsanity.com/paper/2301.12503/) (2023)
- [SDR - half-baked or well done?](https://mlsanity.com/paper/1811.02508/) (2018)
- [Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separation](https://mlsanity.com/paper/1809.07454/) (2018)
- [WaveGlow: A Flow-based Generative Network for Speech Synthesis](https://mlsanity.com/paper/1811.00002/) (2018)
- [Speech Enhancement and Dereverberation with Diffusion-based Generative Models](https://mlsanity.com/paper/2208.05830/) (2022)
- [It's Raw! Audio Generation with State-Space Models](https://mlsanity.com/paper/2202.09729/) (2022)
- [High Fidelity Speech Synthesis with Adversarial Networks](https://mlsanity.com/paper/1909.11646/) (2019)
- [Universal Speech Enhancement with Score-based Diffusion](https://mlsanity.com/paper/2206.03065/) (2022)
- [Non-Attentive Tacotron: Robust and Controllable Neural TTS Synthesis Including Unsupervised Duration Modeling](https://mlsanity.com/paper/2010.04301/) (2020)
- [Solving Audio Inverse Problems with a Diffusion Model](https://mlsanity.com/paper/2210.15228/) (2022)
- [VoiceFixer: Toward General Speech Restoration with Neural Vocoder](https://mlsanity.com/paper/2109.13731/) (2021)
- [HiFi++: a Unified Framework for Bandwidth Extension and Speech Enhancement](https://mlsanity.com/paper/2203.13086/) (2022)
- [Diffusion-based Generative Speech Source Separation](https://mlsanity.com/paper/2210.17327/) (2022)
- [CRASH: Raw Audio Score-based Generative Modeling for Controllable High-resolution Drum Sound Synthesis](https://mlsanity.com/paper/2106.07431/) (2021)
- [Temporal FiLM: Capturing Long-Range Sequence Dependencies with Feature-Wise Modulations](https://mlsanity.com/paper/1909.06628/) (2019)
- [Audio Super Resolution using Neural Networks](https://mlsanity.com/paper/1708.00853/) (2017)
- [FFC-SE: Fast Fourier Convolution for Speech Enhancement](https://mlsanity.com/paper/2204.03042/) (2022)
- [A Proper version of Synthesis-based Sparse Audio Declipper](https://mlsanity.com/paper/1810.12204/) (2018)

## Built on by (most cited first, 7 of 7)

- [Investigating the Design Space of Diffusion Models for Speech Enhancement](https://mlsanity.com/paper/2312.04370/) (2023)
- [Solving Diffusion Inverse Problems with Restart Posterior Sampling](https://mlsanity.com/paper/2511.20705/) (2025)
- [Unsupervised Single-Channel Audio Separation with Diffusion Source Priors](https://mlsanity.com/paper/2512.07226/) (2025)
- [Posterior Transition Modeling for Unsupervised Diffusion-Based Speech Enhancement](https://mlsanity.com/paper/2507.02391/) (2025)
- [Unsupervised Single-Channel Speech Separation with Diffusion under Speaker-Embedding Guidance](https://mlsanity.com/paper/2509.24395/) (2025)
- [GDiffuSE: Diffusion-based speech enhancement with noise model guidance](https://mlsanity.com/paper/2510.04157/) (2025)
- [Exploring Efficient Waveform Diffusion Models for Foley Sound Generation](https://mlsanity.com/paper/2607.29148/) (2026)

## Abstract (arXiv)

This paper introduces UnDiff, a diffusion probabilistic model capable of solving various speech inverse tasks. Being once trained for speech waveform generation in an unconditional manner, it can be adapted to different tasks including degradation inversion, neural vocoding, and source separation. In this paper, we, first, tackle the challenging problem of unconditional waveform generation by comparing different neural architectures and preconditioning domains. After that, we demonstrate how the trained unconditional diffusion could be adapted to different tasks of speech processing by the means of recent developments in post-training conditioning of diffusion models. Finally, we demonstrate the performance of the proposed technique on the tasks of bandwidth extension, declipping, vocoding, and speech source separation and compare it to the baselines. The codes are publicly available.