# Unsupervised Single-Channel Audio Separation with Diffusion Source Priors

arXiv: https://arxiv.org/abs/2512.07226 · HTML: https://mlsanity.com/paper/2512.07226/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Audio, speech and music, one of 14 directions in Diffusion models.
- Builds on 29 of its 35 references that are in the map.
- First posted on arXiv 8 Dec 2025.

## Placed in

- Diffusion models / [Audio, speech and music](https://mlsanity.com/diffusion/audio-speech-and-music/)

## Builds on (most cited first, 29 of 29)

- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [GLU Variants Improve Transformer](https://mlsanity.com/paper/2002.05202/) (2020)
- [Diffusion Posterior Sampling for General Noisy Inverse Problems](https://mlsanity.com/paper/2209.14687/) (2022)
- [DiffWave: A Versatile Diffusion Model for Audio Synthesis](https://mlsanity.com/paper/2009.09761/) (2020)
- [Gradient Surgery for Multi-Task Learning](https://mlsanity.com/paper/2001.06782/) (2020)
- [Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separation](https://mlsanity.com/paper/1809.07454/) (2018)
- [Come-Closer-Diffuse-Faster: Accelerating Conditional Diffusion Models for Inverse Problems through Stochastic Contraction](https://mlsanity.com/paper/2112.05146/) (2021)
- [A Survey on Generative Diffusion Model](https://mlsanity.com/paper/2209.02646/) (2022)
- [Generative Diffusion Prior for Unified Image Restoration and Enhancement](https://mlsanity.com/paper/2304.01247/) (2023)
- [Permutation Invariant Training of Deep Models for Speaker-Independent Multi-talker Speech Separation](https://mlsanity.com/paper/1607.00325/) (2016)
- [Dual-path RNN: efficient long sequence modeling for time-domain single-channel speech separation](https://mlsanity.com/paper/1910.06379/) (2019)
- [A Survey on Diffusion Models for Inverse Problems](https://mlsanity.com/paper/2410.00083/) (2024)
- [Guidance with Spherical Gaussian Constraint for Conditional Diffusion](https://mlsanity.com/paper/2402.03201/) (2024)
- [TF-GridNet: Integrating Full- and Sub-Band Modeling for Speech Separation](https://mlsanity.com/paper/2211.12433/) (2022)
- [Multi-Source Diffusion Models for Simultaneous Music Generation and Separation](https://mlsanity.com/paper/2302.02257/) (2023)
- [U-DiTs: Downsample Tokens in U-Shaped Diffusion Transformers](https://mlsanity.com/paper/2405.02730/) (2024)
- [An efficient encoder-decoder architecture with top-down attention for speech separation](https://mlsanity.com/paper/2209.15200/) (2022)
- [Source Separation with Deep Generative Priors](https://mlsanity.com/paper/2002.07942/) (2020)
- [General-purpose Tagging of Freesound Audio with AudioSet Labels: Task Description, Dataset, and Baseline](https://mlsanity.com/paper/1807.09902/) (2018)
- [Accelerating Diffusion Models for Inverse Problems through Shortcut Sampling](https://mlsanity.com/paper/2305.16965/) (2023)
- [Unsupervised Audio Source Separation using Generative Priors](https://mlsanity.com/paper/2005.13769/) (2020)
- [Generative Speech Foundation Model Pretraining for High-Quality Speech Extraction and Restoration](https://mlsanity.com/paper/2409.16117/) (2024)
- [TIGER: Time-frequency Interleaved Gain Extraction and Reconstruction for Efficient Speech Separation](https://mlsanity.com/paper/2410.01469/) (2024)
- [Music Source Separation with Generative Flow](https://mlsanity.com/paper/2204.09079/) (2022)
- [VoiceDiT: Dual-Condition Diffusion Transformer for Environment-Aware Speech Synthesis](https://mlsanity.com/paper/2412.19259/) (2024)
- [CLAPSep: Leveraging Contrastive Pre-trained Model for Multi-Modal Query-Conditioned Target Sound Extraction](https://mlsanity.com/paper/2402.17455/) (2024)
- [Audio Source Separation Using Variational Autoencoders and Weak Class Supervision](https://mlsanity.com/paper/1810.13104/) (2018)
- [UnDiff: Unsupervised Voice Restoration with Unconditional Diffusion Model](https://mlsanity.com/paper/2306.00721/) (2023)
- [Distance Based Single-Channel Target Speech Extraction](https://mlsanity.com/paper/2412.20144/) (2024)

## Built on by (most cited first, 3 of 3)

- [Joint Residual Reweighting for Classifier Free Guidance in Flow-Matching Zero-Shot TTS](https://mlsanity.com/paper/2606.25672/) (2026)
- [SURF: Separation via Unsupervised Remixing Flow](https://mlsanity.com/paper/2606.04921/) (2026)
- [Exploring Efficient Waveform Diffusion Models for Foley Sound Generation](https://mlsanity.com/paper/2607.29148/) (2026)

## Abstract (arXiv)

Single-channel audio separation aims to separate individual sources from a single-channel mixture. Most existing methods rely on supervised learning with synthetically generated paired data. However, obtaining high-quality paired data in real-world scenarios is often difficult. This data scarcity can degrade model performance under unseen conditions and limit generalization ability. To this end, in this work, we approach this problem from an unsupervised perspective, framing it as a probabilistic inverse problem. Our method requires only diffusion priors trained on individual sources. Separation is then achieved by iteratively guiding an initial state toward the solution through reconstruction guidance. Importantly, we introduce an advanced inverse problem solver specifically designed for separation, which mitigates gradient conflicts caused by interference between the diffusion prior and reconstruction guidance during inverse denoising. This design ensures high-quality and balanced separation performance across individual sources. Additionally, we find that initializing the denoising process with an augmented mixture instead of pure Gaussian noise provides an informative starting point that significantly improves the final performance. To further enhance audio prior modeling, we design a novel time-frequency attention-based network architecture that demonstrates strong audio modeling capability. Collectively, these improvements lead to significant performance gains, as validated across speech-sound event, sound event, and speech separation tasks.