# Music Source Separation in the Waveform Domain

arXiv: https://arxiv.org/abs/1911.13254 · HTML: https://mlsanity.com/paper/1911.13254/

## Key facts

- Cited by 99 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Nov 2019.

## Built on by (most cited first, 30 of 99)

- [High Fidelity Neural Audio Compression](https://mlsanity.com/paper/2210.13438/) (2022)
- [Simple and Controllable Music Generation](https://mlsanity.com/paper/2306.05284/) (2023)
- [Hybrid Spectrogram and Waveform Source Separation](https://mlsanity.com/paper/2111.03600/) (2021)
- [MiniCPM-o 4.5: Towards Real-Time Full-Duplex Omni-Modal Interaction](https://mlsanity.com/paper/2604.27393/) (2026)
- [Music Source Separation with Band-split RNN](https://mlsanity.com/paper/2209.15174/) (2022)
- [Multi-Source Diffusion Models for Simultaneous Music Generation and Separation](https://mlsanity.com/paper/2302.02257/) (2023)
- [HiFi++: a Unified Framework for Bandwidth Extension and Speech Enhancement](https://mlsanity.com/paper/2203.13086/) (2022)
- [Differentiable Model Compression via Pseudo Quantization Noise](https://mlsanity.com/paper/2104.09987/) (2021)
- [VidMuse: A Simple Video-to-Music Generation Framework with Long-Short-Term Modeling](https://mlsanity.com/paper/2406.04321/) (2024)
- [Foundation Models for Music: A Survey](https://mlsanity.com/paper/2408.14340/) (2024)
- [From Discrete Tokens to High-Fidelity Audio Using Multi-Band Diffusion](https://mlsanity.com/paper/2308.02560/) (2023)
- [Parallel and Flexible Sampling from Autoregressive Models via Langevin Dynamics](https://mlsanity.com/paper/2105.08164/) (2021)
- [On loss functions and evaluation metrics for music source separation](https://mlsanity.com/paper/2202.07968/) (2022)
- [Shepherding Slots to Objects: Towards Stable and Robust Object-Centric Learning](https://mlsanity.com/paper/2303.17842/) (2023)
- [Compute and memory efficient universal sound source separation](https://mlsanity.com/paper/2103.02644/) (2021)
- [Text-to-Song: Towards Controllable Music Generation Incorporating Vocals and Accompaniment](https://mlsanity.com/paper/2404.09313/) (2024)
- [Simultaneous Music Separation and Generation Using Multi-Track Latent Diffusion Models](https://mlsanity.com/paper/2409.12346/) (2024)
- [Latent Autoregressive Source Separation](https://mlsanity.com/paper/2301.08562/) (2023)
- [Music Enhancement via Image Translation and Vocoding](https://mlsanity.com/paper/2204.13289/) (2022)
- [The Chamber Ensemble Generator: Limitless High-Quality MIR Data via Generative Modeling](https://mlsanity.com/paper/2209.14458/) (2022)
- [FunnyNet-W: Multimodal Learning of Funny Moments in Videos in the Wild](https://mlsanity.com/paper/2401.04210/) (2024)
- [Hybrid Neural Networks for On-device Directional Hearing](https://mlsanity.com/paper/2112.05893/) (2021)
- [Music Separation Enhancement with Generative Modeling](https://mlsanity.com/paper/2208.12387/) (2022)
- [A Generalized Bandsplit Neural Network for Cinematic Audio Source Separation](https://mlsanity.com/paper/2309.02539/) (2023)
- [Implicit Neural Spatial Filtering for Multichannel Source Separation in the Waveform Domain](https://mlsanity.com/paper/2206.15423/) (2022)
- [Benchmarks and leaderboards for sound demixing tasks](https://mlsanity.com/paper/2305.07489/) (2023)
- [Listen, Chat, and Remix: Text-Guided Soundscape Remixing for Enhanced Auditory Experience](https://mlsanity.com/paper/2402.03710/) (2024)
- [RF Challenge: The Data-Driven Radio Frequency Signal Separation Challenge](https://mlsanity.com/paper/2409.08839/) (2024)
- [Score Distillation Sampling for Audio: Source Separation, Synthesis, and Beyond](https://mlsanity.com/paper/2505.04621/) (2025)
- [MaineCoon: Pursuing A Real-Time Audio-Visual Social World Model](https://mlsanity.com/paper/2606.17800/) (2026)

## Abstract (arXiv)

Source separation for music is the task of isolating contributions, or stems, from different instruments recorded individually and arranged together to form a song. Such components include voice, bass, drums and any other accompaniments.Contrarily to many audio synthesis tasks where the best performances are achieved by models that directly generate the waveform, the state-of-the-art in source separation for music is to compute masks on the magnitude spectrum. In this paper, we compare two waveform domain architectures. We first adapt Conv-Tasnet, initially developed for speech source separation,to the task of music source separation. While Conv-Tasnet beats many existing spectrogram-domain methods, it suffersfrom significant artifacts, as shown by human evaluations. We propose instead Demucs, a novel waveform-to-waveform model,with a U-Net structure and bidirectional LSTM.Experiments on the MusDB dataset show that, with proper data augmentation, Demucs beats allexisting state-of-the-art architectures, including Conv-Tasnet, with 6.3 SDR on average, (and up to 6.8 with 150 extra training songs, even surpassing the IRM oracle for the bass source).Using recent development in model quantization, Demucs can be compressed down to 120MBwithout any loss of accuracy.We also provide human evaluations, showing that Demucs benefit from a large advantagein terms of the naturalness of the audio. However, it suffers from some bleeding,especially between the vocals and other source.