# Diff-VS: Efficient Audio-Aware Diffusion U-Net for Vocals Separation

arXiv: https://arxiv.org/abs/2604.01120 · HTML: https://mlsanity.com/paper/2604.01120/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Audio, speech and music, one of 14 directions in Diffusion models.
- Builds on 28 of its 31 references that are in the map.
- First posted on arXiv 1 Apr 2026.

## Placed in

- Diffusion models / [Audio, speech and music](https://mlsanity.com/diffusion/audio-speech-and-music/)

## Builds on (most cited first, 28 of 28)

- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [High Fidelity Neural Audio Compression](https://mlsanity.com/paper/2210.13438/) (2022)
- [Speech Enhancement and Dereverberation with Diffusion-based Generative Models](https://mlsanity.com/paper/2208.05830/) (2022)
- [Stable Audio Open](https://mlsanity.com/paper/2407.14358/) (2024)
- [Conditional Diffusion Probabilistic Model for Speech Enhancement](https://mlsanity.com/paper/2202.05256/) (2022)
- [Wave-U-Net: A Multi-Scale Neural Network for End-to-End Audio Source Separation](https://mlsanity.com/paper/1806.03185/) (2018)
- [Speech Enhancement with Score-Based Generative Models in the Complex STFT Domain](https://mlsanity.com/paper/2203.17004/) (2022)
- [Hybrid Spectrogram and Waveform Source Separation](https://mlsanity.com/paper/2111.03600/) (2021)
- [Universal Speech Enhancement with Score-based Diffusion](https://mlsanity.com/paper/2206.03065/) (2022)
- [Music Source Separation with Band-split RNN](https://mlsanity.com/paper/2209.15174/) (2022)
- [Multi-Source Diffusion Models for Simultaneous Music Generation and Separation](https://mlsanity.com/paper/2302.02257/) (2023)
- [The 2018 Signal Separation Evaluation Campaign](https://mlsanity.com/paper/1804.06267/) (2018)
- [Moisesdb: A dataset for source separation beyond 4-stems](https://mlsanity.com/paper/2307.15913/) (2023)
- [Music Source Separation with Band-Split RoPE Transformer](https://mlsanity.com/paper/2309.02612/) (2023)
- [Diff-A-Riff: Musical Accompaniment Co-creation via Latent Diffusion Models](https://mlsanity.com/paper/2406.08384/) (2024)
- [Investigating Training Objectives for Generative Speech Enhancement](https://mlsanity.com/paper/2409.10753/) (2024)
- [A consolidated view of loss functions for supervised deep learning-based speech enhancement](https://mlsanity.com/paper/2009.12286/) (2020)
- [ETTA: Elucidating the Design Space of Text-to-Audio Models](https://mlsanity.com/paper/2412.19351/) (2024)
- [Simultaneous Music Separation and Generation Using Multi-Track Latent Diffusion Models](https://mlsanity.com/paper/2409.12346/) (2024)
- [Adversarial Semi-Supervised Audio Source Separation applied to Singing Voice Extraction](https://mlsanity.com/paper/1711.00048/) (2017)
- [SCNet: Sparse Compression Network for Music Source Separation](https://mlsanity.com/paper/2401.13276/) (2024)
- [Mel-RoFormer for Vocal Separation and Vocal Melody Transcription](https://mlsanity.com/paper/2409.04702/) (2024)
- [Sound Demixing Challenge 2023 Music Demixing Track Technical Report: TFC-TDF-UNet v3](https://mlsanity.com/paper/2306.09382/) (2023)
- [Towards Reliable Objective Evaluation Metrics for Generative Singing Voice Separation Models](https://mlsanity.com/paper/2507.11427/) (2025)
- [Efficient and Fast Generative-Based Singing Voice Separation using a Latent Diffusion Model](https://mlsanity.com/paper/2511.20470/) (2025)
- [Moises-Light: Resource-efficient Band-split U-Net For Music Source Separation](https://mlsanity.com/paper/2510.06785/) (2025)
- [Generating Separated Singing Vocals Using a Diffusion Model Conditioned on Music Mixtures](https://mlsanity.com/paper/2511.21342/) (2025)

## Abstract (arXiv)

While diffusion models are best known for their performance in generative tasks, they have also been successfully applied to many other tasks, including audio source separation. However, current generative approaches to music source separation often underperform on standard objective metrics. In this paper, we address this issue by introducing a novel generative vocal separation model based on the Elucidated Diffusion Model (EDM) framework. Our model processes complex short-time Fourier transform spectrograms and employs an improved U-Net architecture based on music-informed design choices. Our approach matches discriminative baselines on objective metrics and achieves perceptual quality comparable to state-of-the-art systems, as assessed by proxy subjective metrics. We hope these results encourage broader exploration of generative methods for music source separation