# Unsupervised Audio Source Separation using Generative Priors

arXiv: https://arxiv.org/abs/2005.13769 · HTML: https://mlsanity.com/paper/2005.13769/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 May 2020.

## Built on by (most cited first, 17 of 17)

- [Multi-Source Diffusion Models for Simultaneous Music Generation and Separation](https://mlsanity.com/paper/2302.02257/) (2023)
- [Simultaneous Music Separation and Generation Using Multi-Track Latent Diffusion Models](https://mlsanity.com/paper/2409.12346/) (2024)
- [Latent Autoregressive Source Separation](https://mlsanity.com/paper/2301.08562/) (2023)
- [Adversarial Permutation Invariant Training for Universal Sound Separation](https://mlsanity.com/paper/2210.12108/) (2022)
- [Unsupervised Music Source Separation Using Differentiable Parametric Source Models](https://mlsanity.com/paper/2201.09592/) (2022)
- [Differentiable Digital Signal Processing Mixture Model for Synthesis Parameter Extraction from Mixture of Harmonic Sounds](https://mlsanity.com/paper/2202.00200/) (2022)
- [MGE-LDM: Joint Latent Diffusion for Simultaneous Music Generation and Source Extraction](https://mlsanity.com/paper/2505.23305/) (2025)
- [Unsupervised Single-Channel Audio Separation with Diffusion Source Priors](https://mlsanity.com/paper/2512.07226/) (2025)
- [On the Design of Deep Priors for Unsupervised Audio Restoration](https://mlsanity.com/paper/2104.07161/) (2021)
- [Unsupervised Source Separation via Bayesian Inference in the Latent Domain](https://mlsanity.com/paper/2110.05313/) (2021)
- [Efficient and Fast Generative-Based Singing Voice Separation using a Latent Diffusion Model](https://mlsanity.com/paper/2511.20470/) (2025)
- [Distribution Preserving Source Separation With Time Frequency Predictive Models](https://mlsanity.com/paper/2303.05896/) (2023)
- [Improving Music Source Separation with Diffusion and Consistency Refinement](https://mlsanity.com/paper/2412.06965/) (2024)
- [Generating Separated Singing Vocals Using a Diffusion Model Conditioned on Music Mixtures](https://mlsanity.com/paper/2511.21342/) (2025)
- [Blind Source Separation in Polyphonic Music Recordings Using Deep Neural Networks Trained via Policy Gradients](https://mlsanity.com/paper/2107.04235/) (2021)
- [Semi-Blind Source Separation with Learned Constraints](https://mlsanity.com/paper/2209.13585/) (2022)
- [Unsupervised Single-Channel Speech Separation with Diffusion under Speaker-Embedding Guidance](https://mlsanity.com/paper/2509.24395/) (2025)

## Abstract (arXiv)

State-of-the-art under-determined audio source separation systems rely on supervised end-end training of carefully tailored neural network architectures operating either in the time or the spectral domain. However, these methods are severely challenged in terms of requiring access to expensive source level labeled data and being specific to a given set of sources and the mixing process, which demands complete re-training when those assumptions change. This strongly emphasizes the need for unsupervised methods that can leverage the recent advances in data-driven modeling, and compensate for the lack of labeled data through meaningful priors. To this end, we propose a novel approach for audio source separation based on generative priors trained on individual sources. Through the use of projected gradient descent optimization, our approach simultaneously searches in the source-specific latent spaces to effectively recover the constituent sources. Though the generative priors can be defined in the time domain directly, e.g. WaveGAN, we find that using spectral domain loss functions for our optimization leads to good-quality source estimates. Our empirical studies on standard spoken digit and instrument datasets clearly demonstrate the effectiveness of our approach over classical as well as state-of-the-art unsupervised baselines.