# Conditioned Source Separation for Music Instrument Performances

arXiv: https://arxiv.org/abs/2004.03873 · HTML: https://mlsanity.com/paper/2004.03873/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Apr 2020.

## Built on by (most cited first, 18 of 18)

- [VoViT: Low Latency Graph-based Audio-Visual Voice Separation Transformer](https://mlsanity.com/paper/2203.04099/) (2022)
- [Heterogeneous Target Speech Separation](https://mlsanity.com/paper/2204.03594/) (2022)
- [A cappella: Audio-visual Singing Voice Separation](https://mlsanity.com/paper/2104.09946/) (2021)
- [DPM-TSE: A Diffusion Probabilistic Model for Target Sound Extraction](https://mlsanity.com/paper/2310.04567/) (2023)
- [Multistage linguistic conditioning of convolutional layers for speech emotion recognition](https://mlsanity.com/paper/2110.06650/) (2021)
- [A Stem-Agnostic Single-Decoder System for Music Source Separation Beyond Four Stems](https://mlsanity.com/paper/2406.18747/) (2024)
- [Optimal Condition Training for Target Source Separation](https://mlsanity.com/paper/2211.05927/) (2022)
- [SynthSOD: Developing an Heterogeneous Dataset for Orchestra Music Source Separation](https://mlsanity.com/paper/2409.10995/) (2024)
- [Music Audio-Visual Question Answering Requires Specialized Multimodal Designs](https://mlsanity.com/paper/2505.20638/) (2025)
- [Music source separation conditioned on 3D point clouds](https://mlsanity.com/paper/2102.02028/) (2021)
- [Score-informed Music Source Separation: Improving Synthetic-to-real Generalization in Classical Music](https://mlsanity.com/paper/2503.07352/) (2025)
- [Locate This, Not That: Class-Conditioned Sound Event DOA Estimation](https://mlsanity.com/paper/2203.04197/) (2022)
- [The Spheres Dataset: Multitrack Orchestral Recordings for Music Source Separation and Information Retrieval](https://mlsanity.com/paper/2511.21247/) (2025)
- [LightSAFT: Lightweight Latent Source Aware Frequency Transform for Source Separation](https://mlsanity.com/paper/2111.12516/) (2021)
- [SANE-TTS: Stable And Natural End-to-End Multilingual Text-to-Speech](https://mlsanity.com/paper/2206.12132/) (2022)
- [Pre-trained Spatial Priors on Multichannel NMF for Music Source Separation](https://mlsanity.com/paper/2310.05821/) (2023)
- [CATSE: A Context-Aware Framework for Causal Target Sound Extraction](https://mlsanity.com/paper/2403.14246/) (2024)
- [A Conditioned UNet for Music Source Separation](https://mlsanity.com/paper/2512.15532/) (2025)

## Abstract (arXiv)

In music source separation, the number of sources may vary for each piece and some of the sources may belong to the same family of instruments, thus sharing timbral characteristics and making the sources more correlated. This leads to additional challenges in the source separation problem. This paper proposes a source separation method for multiple musical instruments sounding simultaneously and explores how much additional information apart from the audio stream can lift the quality of source separation. We explore conditioning techniques at different levels of a primary source separation network and utilize two extra modalities of data, namely presence or absence of instruments in the mixture, and the corresponding video stream data.