# A Stem-Agnostic Single-Decoder System for Music Source Separation Beyond Four Stems

arXiv: https://arxiv.org/abs/2406.18747 · HTML: https://mlsanity.com/paper/2406.18747/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 51 references that are in the map.
- First posted on arXiv 26 Jun 2024.

## Builds on (most cited first, 30 of 38)

- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [SDR - half-baked or well done?](https://mlsanity.com/paper/1811.02508/) (2018)
- [Efficient Training of Audio Transformers with Patchout](https://mlsanity.com/paper/2110.05069/) (2021)
- [Hybrid Transformers for Music Source Separation](https://mlsanity.com/paper/2211.08553/) (2022)
- [Wave-U-Net: A Multi-Scale Neural Network for End-to-End Audio Source Separation](https://mlsanity.com/paper/1806.03185/) (2018)
- [Hybrid Spectrogram and Waveform Source Separation](https://mlsanity.com/paper/2111.03600/) (2021)
- [Cutting Music Source Separation Some Slakh: A Dataset to Study the Impact of Training Data Quality and Quantity](https://mlsanity.com/paper/1909.08494/) (2019)
- [Music Source Separation with Band-split RNN](https://mlsanity.com/paper/2209.15174/) (2022)
- [Creating A Multi-track Classical Musical Performance Dataset for Multimodal Music Analysis: Challenges, Insights, and Applications](https://mlsanity.com/paper/1612.08727/) (2016)
- [Demucs: Deep Extractor for Music Sources with extra unlabeled data remixed](https://mlsanity.com/paper/1909.01174/) (2019)
- [Decoupling Magnitude and Phase Estimation with Deep ResUNet for Music Source Separation](https://mlsanity.com/paper/2109.05418/) (2021)
- [Separate What You Describe: Language-Queried Audio Source Separation](https://mlsanity.com/paper/2203.15147/) (2022)
- [Separate Anything You Describe](https://mlsanity.com/paper/2308.05037/) (2023)
- [The 2018 Signal Separation Evaluation Campaign](https://mlsanity.com/paper/1804.06267/) (2018)
- [Moisesdb: A dataset for source separation beyond 4-stems](https://mlsanity.com/paper/2307.15913/) (2023)
- [Music Demixing Challenge 2021](https://mlsanity.com/paper/2108.13559/) (2021)
- [Music Source Separation with Band-Split RoPE Transformer](https://mlsanity.com/paper/2309.02612/) (2023)
- [Conditioned-U-Net: Introducing a Control Mechanism in the U-Net for Multiple Source Separations](https://mlsanity.com/paper/1907.01277/) (2019)
- [Zero-shot Audio Source Separation through Query-based Learning from Weakly-labeled Data](https://mlsanity.com/paper/2112.07891/) (2021)
- [D3Net: Densely connected multidilated DenseNet for music source separation](https://mlsanity.com/paper/2010.01733/) (2020)
- [KUIELab-MDX-Net: A Two-Stream Neural Network for Music Demixing](https://mlsanity.com/paper/2111.12203/) (2021)
- [The Sound Demixing Challenge 2023 unicodex2013 Music Demixing Track](https://mlsanity.com/paper/2308.06979/) (2023)
- [The Cocktail Fork Problem: Three-Stem Audio Separation for Real-World Soundtracks](https://mlsanity.com/paper/2110.09958/) (2021)
- [Meta-learning Extractors for Music Source Separation](https://mlsanity.com/paper/2002.07016/) (2020)
- [Conditioned Source Separation for Music Instrument Performances](https://mlsanity.com/paper/2004.03873/) (2020)
- [One-shot conditional audio filtering of arbitrary sounds](https://mlsanity.com/paper/2011.02421/) (2020)
- [A Unified Model for Zero-shot Music Source Separation, Transcription and Synthesis](https://mlsanity.com/paper/2108.03456/) (2021)
- [Audio query-based music source separation](https://mlsanity.com/paper/1908.06593/) (2019)
- [Multi-scale Multi-band DenseNets for Audio Source Separation](https://mlsanity.com/paper/1706.09588/) (2017)
- [LaSAFT: Latent Source Attentive Frequency Transformation for Conditioned Source Separation](https://mlsanity.com/paper/2010.11631/) (2020)

## Built on by (most cited first, 7 of 7)

- [User-guided Generative Source Separation](https://mlsanity.com/paper/2507.01339/) (2025)
- [AI-Assisted Music Production: A User Study on Text-to-Music Models](https://mlsanity.com/paper/2509.23364/) (2025)
- [Facing the Music: Tackling Singing Voice Separation in Cinematic Audio Source Separation](https://mlsanity.com/paper/2408.03588/) (2024)
- [The Perception of Phase Intercept Distortion and its Application in Data Augmentation](https://mlsanity.com/paper/2506.14571/) (2025)
- [SoundCompass: Navigating Target Sound Extraction With Effective Directional Clue Integration In Complex Acoustic Scenes](https://mlsanity.com/paper/2509.18561/) (2025)
- [Transcribing Rhythmic Patterns of the Guitar Track in Polyphonic Music](https://mlsanity.com/paper/2510.05756/) (2025)
- [A Conditioned UNet for Music Source Separation](https://mlsanity.com/paper/2512.15532/) (2025)

## Abstract (arXiv)

Despite significant recent progress across multiple subtasks of audio source separation, few music source separation systems support separation beyond the four-stem vocals, drums, bass, and other (VDBO) setup. Of the very few current systems that support source separation beyond this setup, most continue to rely on an inflexible decoder setup that can only support a fixed pre-defined set of stems. Increasing stem support in these inflexible systems correspondingly requires increasing computational complexity, rendering extensions of these systems computationally infeasible for long-tail instruments. In this work, we propose Banquet, a system that allows source separation of multiple stems using just one decoder. A bandsplit source separation model is extended to work in a query-based setup in tandem with a music instrument recognition PaSST model. On the MoisesDB dataset, Banquet, at only 24.9 M trainable parameters, approached the performance level of the significantly more complex 6-stem Hybrid Transformer Demucs on VDBO stems and outperformed it on guitar and piano. The query-based setup allows for the separation of narrow instrument classes such as clean acoustic guitars, and can be successfully applied to the extraction of less common stems such as reeds and organs. Implementation is available at https://github.com/kwatcharasupat/query-bandit.