Demucs: Deep Extractor for Music Sources with extra unlabeled data remixed
Not on the map. Why?
Abstract (arXiv)
We study the problem of source separation for music using deep learning with four known sources: drums, bass, vocals and other accompaniments. State-of-the-art approaches predict soft masks over mixture spectrograms while methods working on the waveform are lagging behind as measured on the standard MusDB benchmark. Our contribution is two fold. (i) We introduce a simple convolutional and recurrent model that outperforms the state-of-the-art model on waveforms, that is, Wave-U-Net, by 1.6 points of SDR (signal to distortion ratio). (ii) We propose a new scheme to leverage unlabeled music. We train a first model to extract parts with at least one source silent in unlabeled tracks, for instance without bass. We remix this extract with a bass line taken from the supervised dataset to form a new weakly supervised training example. Combining our architecture and scheme, we show that waveform methods can play in the same ballpark as spectrogram ones.
Key facts. Cited by 53 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Sep 2019.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 7; 2022: 5; 2023: 6; 2024: 9; 2025: 15; 2026: 11.
53 papers.
Sort:
By area:
- 2022Music Source Separation with Band-split RNNcited by 69
- 2023Separate Anything You Describecited by 43
- 2021Zero-shot Audio Source Separation through Query-based Learning from Weakly-labeled Datacited by 27
- 2025IndexTTS: An Industrial-Level Controllable and Efficient Zero-Shot Text-To-Speech Systemcited by 27
- 2023MusicAgent: An AI Agent for Music Understanding and Generation with Large Language Modelscited by 20
- 2023Content-based Controls For Music Large Language Modelingcited by 19
- 2025LLaSE-G1: Incentivizing Generalization Capability for LLaMA-based Speech Enhancementcited by 19
- 2025Analyzable Chain-of-Musical-Thought Prompting for High-Fidelity Music Generationcited by 13
- 2023ReZero: Region-customizable Sound Extractioncited by 10
- 2021CWS-PResUNet: Music Source Separation with Channel-wise Subband Phase-aware ResUNetcited by 9
- 2024OmniSep: Unified Omni-Modality Sound Separation with Query-Mixupcited by 8
- 2026MSAVBench: Towards Comprehensive and Reliable Evaluation of Multi-Shot Audio-Video Generationcited by 8
38 more
- 2024A Stem-Agnostic Single-Decoder System for Music Source Separation Beyond Four Stemscited by 7
- 2024SaMoye: Zero-shot Singing Voice Conversion Model Based on Feature Disentanglement and Enhancementcited by 5
- 2021CycleDRUMS: Automatic Drum Arrangement For Bass Lines Using CycleGANcited by 4
- 2021Transfer Learning with Jukebox for Music Source Separationcited by 4
- 2022Unsupervised Music Source Separation Using Differentiable Parametric Source Modelscited by 4
- 2024FastSAG: Towards Fast Non-Autoregressive Singing Accompaniment Generationcited by 4
- 2024MUSE: Flexible Voiceprint Receptive Fields and Multi-Path Fusion Enhanced Taylor Transformer for U-Net-based Speech Enhancementcited by 4
- 2025Adaptive Slimming for Scalable and Efficient Speech Enhancementcited by 4
- 2025UniFlow: Unifying Speech Front-End Tasks via Continuous Generative Modelingcited by 4
- 2021Audiovisual Singing Voice Separationcited by 3
- 2023Looking Similar, Sounding Different: Leveraging Counterfactual Cross-Modal Pairs for Audiovisual Representation Learningcited by 3
- 2025SAO-Instruct: Free-form Audio Editing using Natural Language Instructionscited by 3
- 2021On the Design of Deep Priors for Unsupervised Audio Restorationcited by 2
- 2022Improved singing voice separation with chromagram-based pitch-aware remixingcited by 2
- 2024Subtractive Training for Music Stem Insertion using Latent Diffusion Modelscited by 2
- 2024U-Mamba-Net: A highly efficient Mamba-based U-net style network for noisy and reverberant speech separationcited by 2
- 2025Deepfake Detection of Singing Voices With Whisper Encodingscited by 2
- 2025Integrating Feedback Loss from Bi-modal Sarcasm Detector for Sarcastic Speech Synthesiscited by 2
- 2026The Radio-Frequency Transformer for Signal Separationcited by 2
- 2021Using Python for Model Inference in Deep Learningcited by 1
- 2022Audio Denoising for Robust Audio Fingerprintingcited by 1
- 2024An Ensemble Approach to Music Source Separation: A Comparative Analysis of Conventional and Hierarchical Stem Separationcited by 1
- 2025Unleashing the Power of Natural Audio Featuring Multiple Sound Sourcescited by 1
- 2025Towards Hallucination-Free Music: A Reinforcement Learning Preference Optimization Framework for Reliable Song Generationcited by 1
- 2025Optimizing Neural Architectures for Hindi Speech Separation and Enhancement in Noisy Environmentscited by 1
- 2025AI-Assisted Music Production: A User Study on Text-to-Music Modelscited by 1
- 2026Identity as Presence: Towards Appearance and Voice Personalized Joint Audio-Video Generationcited by 1
- 2026OmniHuman: A Large-scale Dataset and Benchmark for Human-Centric Video Generationcited by 1
- 2022GAFX: A General Audio Feature eXtractorcited by 0
- 2023Music Augmentation and Denoising For Peak-Based Audio Fingerprintingcited by 0
- 2024Extract and Diffuse: Latent Integration for Improved Diffusion-based Speech and Vocal Enhancementcited by 0
- 2025Solving Copyright Infringement on Short Video Platforms: Novel Datasets and an Audio Restoration Deep Learning Pipelinecited by 0
- 2025Source Separation for A Cappella Musiccited by 0
- 2025IMSE: Efficient U-Net-based Speech Enhancement using Inception Depthwise Convolution and Amplitude-Aware Linear Attentioncited by 0
- 2026A Knowledge-Driven Approach to Music Segmentation, Music Source Separation and Cinematic Audio Source Separationcited by 0
- 2026An Investigation Into Various Approaches For Bengali Long-Form Speech Transcription and Bengali Speaker Diarizationcited by 0
- 2026S2Accompanist: A Semantic-Aware and Structure-Guided Diffusion Model for Music Accompaniment Generationcited by 0
- 2026Generalized Audio-Driven Synthesis of Precise Drummer Motioncited by 0
50 most cited of 53 All 53 (JSON)
- 2026MambaVoice: Lightweight Audiovisual Singing Voice Separation Via A Hybrid Mamba-Transformer Model22 Sep 2026
- 2026AURA: Unified Multimodal Framework for Conversational Music Editing13 Sep 2026
- 2026Generalized Audio-Driven Synthesis of Precise Drummer Motion19 Aug 2026
- 2026Poly-InstructTTS: Learning In-the-Wild Expressive Speech Synthesis from Open-Ended Instructions30 Jun 2026
- 2026MSAVBench: Towards Comprehensive and Reliable Evaluation of Multi-Shot Audio-Video Generation19 May 2026
- 2026S2Accompanist: A Semantic-Aware and Structure-Guided Diffusion Model for Music Accompaniment Generation17 May 2026
- 2026OmniHuman: A Large-scale Dataset and Benchmark for Human-Centric Video Generation20 Apr 2026
- 2026Identity as Presence: Towards Appearance and Voice Personalized Joint Audio-Video Generation18 Mar 2026
- 2026The Radio-Frequency Transformer for Signal Separation10 Mar 2026
- 2026An Investigation Into Various Approaches For Bengali Long-Form Speech Transcription and Bengali Speaker Diarization3 Mar 2026
- 2026A Knowledge-Driven Approach to Music Segmentation, Music Source Separation and Cinematic Audio Source Separation25 Feb 2026
- 2025IMSE: Efficient U-Net-based Speech Enhancement using Inception Depthwise Convolution and Amplitude-Aware Linear Attention18 Nov 2025
38 more
- 2025SAO-Instruct: Free-form Audio Editing using Natural Language Instructions26 Oct 2025
- 2025Source Separation for A Cappella Music30 Sep 2025
- 2025AI-Assisted Music Production: A User Study on Text-to-Music Models27 Sep 2025
- 2025Integrating Feedback Loss from Bi-modal Sarcasm Detector for Sarcastic Speech Synthesis18 Aug 2025
- 2025Optimizing Neural Architectures for Hindi Speech Separation and Enhancement in Noisy Environments16 Aug 2025
- 2025UniFlow: Unifying Speech Front-End Tasks via Continuous Generative Modeling11 Aug 2025
- 2025Towards Hallucination-Free Music: A Reinforcement Learning Preference Optimization Framework for Reliable Song Generation7 Aug 2025
- 2025Adaptive Slimming for Scalable and Efficient Speech Enhancement7 Jul 2025
- 2025Solving Copyright Infringement on Short Video Platforms: Novel Datasets and an Audio Restoration Deep Learning Pipeline30 Apr 2025
- 2025Unleashing the Power of Natural Audio Featuring Multiple Sound Sources24 Apr 2025
- 2025Analyzable Chain-of-Musical-Thought Prompting for High-Fidelity Music Generation25 Mar 2025
- 2025LLaSE-G1: Incentivizing Generalization Capability for LLaMA-based Speech Enhancement1 Mar 2025
- 2025IndexTTS: An Industrial-Level Controllable and Efficient Zero-Shot Text-To-Speech System8 Feb 2025
- 2025Deepfake Detection of Singing Voices With Whisper Encodings31 Jan 2025
- 2024U-Mamba-Net: A highly efficient Mamba-based U-net style network for noisy and reverberant speech separation24 Dec 2024
- 2024OmniSep: Unified Omni-Modality Sound Separation with Query-Mixup28 Oct 2024
- 2024An Ensemble Approach to Music Source Separation: A Comparative Analysis of Conventional and Hierarchical Stem Separation28 Oct 2024
- 2024Extract and Diffuse: Latent Integration for Improved Diffusion-based Speech and Vocal Enhancement15 Sep 2024
- 2024SaMoye: Zero-shot Singing Voice Conversion Model Based on Feature Disentanglement and Enhancement10 Jul 2024
- 2024Subtractive Training for Music Stem Insertion using Latent Diffusion Models27 Jun 2024
- 2024A Stem-Agnostic Single-Decoder System for Music Source Separation Beyond Four Stems26 Jun 2024
- 2024MUSE: Flexible Voiceprint Receptive Fields and Multi-Path Fusion Enhanced Taylor Transformer for U-Net-based Speech Enhancement7 Jun 2024
- 2024FastSAG: Towards Fast Non-Autoregressive Singing Accompaniment Generation13 May 2024
- 2023Content-based Controls For Music Large Language Modeling26 Oct 2023
- 2023Music Augmentation and Denoising For Peak-Based Audio Fingerprinting20 Oct 2023
- 2023MusicAgent: An AI Agent for Music Understanding and Generation with Large Language Models18 Oct 2023
- 2023ReZero: Region-customizable Sound Extraction31 Aug 2023
- 2023Separate Anything You Describe9 Aug 2023
- 2023Looking Similar, Sounding Different: Leveraging Counterfactual Cross-Modal Pairs for Audiovisual Representation Learning12 Apr 2023
- 2022Audio Denoising for Robust Audio Fingerprinting21 Dec 2022
- 2022Music Source Separation with Band-split RNN30 Sep 2022
- 2022GAFX: A General Audio Feature eXtractor19 Jul 2022
- 2022Improved singing voice separation with chromagram-based pitch-aware remixing28 Mar 2022
- 2022Unsupervised Music Source Separation Using Differentiable Parametric Source Models24 Jan 2022
- 2021Zero-shot Audio Source Separation through Query-based Learning from Weakly-labeled Data15 Dec 2021
- 2021CWS-PResUNet: Music Source Separation with Channel-wise Subband Phase-aware ResUNet9 Dec 2021
- 2021Transfer Learning with Jukebox for Music Source Separation28 Nov 2021
- 2021Audiovisual Singing Voice Separation1 Jul 2021
50 newest of 53 All 53 (JSON)
- 2025UniFlow: Unifying Speech Front-End Tasks via Continuous Generative Modelingcited by 4
- 2024Subtractive Training for Music Stem Insertion using Latent Diffusion Modelscited by 2
- 2024Extract and Diffuse: Latent Integration for Improved Diffusion-based Speech and Vocal Enhancementcited by 0
- 2026S2Accompanist: A Semantic-Aware and Structure-Guided Diffusion Model for Music Accompaniment Generationcited by 0
- 2026S2Accompanist: A Semantic-Aware and Structure-Guided Diffusion Model for Music Accompaniment Generation17 May 2026
- 2025UniFlow: Unifying Speech Front-End Tasks via Continuous Generative Modeling11 Aug 2025
- 2024Extract and Diffuse: Latent Integration for Improved Diffusion-based Speech and Vocal Enhancement15 Sep 2024
- 2024Subtractive Training for Music Stem Insertion using Latent Diffusion Models27 Jun 2024
- 2022Music Source Separation with Band-split RNNcited by 69
- 2023Separate Anything You Describecited by 43
- 2021Zero-shot Audio Source Separation through Query-based Learning from Weakly-labeled Datacited by 27
- 2025IndexTTS: An Industrial-Level Controllable and Efficient Zero-Shot Text-To-Speech Systemcited by 27
- 2023Content-based Controls For Music Large Language Modelingcited by 19
- 2025LLaSE-G1: Incentivizing Generalization Capability for LLaMA-based Speech Enhancementcited by 19
- 2025Analyzable Chain-of-Musical-Thought Prompting for High-Fidelity Music Generationcited by 13
- 2023ReZero: Region-customizable Sound Extractioncited by 10
- 2021CWS-PResUNet: Music Source Separation with Channel-wise Subband Phase-aware ResUNetcited by 9
- 2024OmniSep: Unified Omni-Modality Sound Separation with Query-Mixupcited by 8
- 2026MSAVBench: Towards Comprehensive and Reliable Evaluation of Multi-Shot Audio-Video Generationcited by 8
- 2024A Stem-Agnostic Single-Decoder System for Music Source Separation Beyond Four Stemscited by 7
12 most cited of 45 All 45 (JSON)
- 2026AURA: Unified Multimodal Framework for Conversational Music Editing13 Sep 2026
- 2026Generalized Audio-Driven Synthesis of Precise Drummer Motion19 Aug 2026
- 2026Poly-InstructTTS: Learning In-the-Wild Expressive Speech Synthesis from Open-Ended Instructions30 Jun 2026
- 2026MSAVBench: Towards Comprehensive and Reliable Evaluation of Multi-Shot Audio-Video Generation19 May 2026
- 2026OmniHuman: A Large-scale Dataset and Benchmark for Human-Centric Video Generation20 Apr 2026
- 2026Identity as Presence: Towards Appearance and Voice Personalized Joint Audio-Video Generation18 Mar 2026
- 2026The Radio-Frequency Transformer for Signal Separation10 Mar 2026
- 2026An Investigation Into Various Approaches For Bengali Long-Form Speech Transcription and Bengali Speaker Diarization3 Mar 2026
- 2026A Knowledge-Driven Approach to Music Segmentation, Music Source Separation and Cinematic Audio Source Separation25 Feb 2026
- 2025IMSE: Efficient U-Net-based Speech Enhancement using Inception Depthwise Convolution and Amplitude-Aware Linear Attention18 Nov 2025
- 2025SAO-Instruct: Free-form Audio Editing using Natural Language Instructions26 Oct 2025
- 2025Source Separation for A Cappella Music30 Sep 2025
12 newest of 45 All 45 (JSON)