# Audio query-based music source separation

arXiv: https://arxiv.org/abs/1908.06593 · HTML: https://mlsanity.com/paper/1908.06593/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Aug 2019.

## Built on by (most cited first, 17 of 17)

- [Separate Anything You Describe](https://mlsanity.com/paper/2308.05037/) (2023)
- [iQuery: Instruments as Queries for Audio-Visual Sound Separation](https://mlsanity.com/paper/2212.03814/) (2022)
- [Zero-shot Audio Source Separation through Query-based Learning from Weakly-labeled Data](https://mlsanity.com/paper/2112.07891/) (2021)
- [A Unified Model for Zero-shot Music Source Separation, Transcription and Synthesis](https://mlsanity.com/paper/2108.03456/) (2021)
- [The Chamber Ensemble Generator: Limitless High-Quality MIR Data via Generative Modeling](https://mlsanity.com/paper/2209.14458/) (2022)
- [OmniSep: Unified Omni-Modality Sound Separation with Query-Mixup](https://mlsanity.com/paper/2410.21269/) (2024)
- [A Stem-Agnostic Single-Decoder System for Music Source Separation Beyond Four Stems](https://mlsanity.com/paper/2406.18747/) (2024)
- [MGE-LDM: Joint Latent Diffusion for Simultaneous Music Generation and Source Extraction](https://mlsanity.com/paper/2505.23305/) (2025)
- [Environmental Sound Extraction Using Onomatopoeic Words](https://mlsanity.com/paper/2112.00209/) (2021)
- [DisMix: Disentangling Mixtures of Musical Instruments for Source-level Pitch and Timbre Manipulation](https://mlsanity.com/paper/2408.10807/) (2024)
- [User-guided Generative Source Separation](https://mlsanity.com/paper/2507.01339/) (2025)
- [Locate This, Not That: Class-Conditioned Sound Event DOA Estimation](https://mlsanity.com/paper/2203.04197/) (2022)
- [A Comparative Analysis Of Latent Regressor Losses For Singing Voice Conversion](https://mlsanity.com/paper/2302.13678/) (2023)
- [Universal Sound Separation with Self-Supervised Audio Masked Autoencoder](https://mlsanity.com/paper/2407.11745/) (2024)
- [MARS-Sep: Multimodal-Aligned Reinforced Sound Separation](https://mlsanity.com/paper/2510.10509/) (2025)
- [LightSAFT: Lightweight Latent Source Aware Frequency Transform for Source Separation](https://mlsanity.com/paper/2111.12516/) (2021)
- [A Conditioned UNet for Music Source Separation](https://mlsanity.com/paper/2512.15532/) (2025)

## Abstract (arXiv)

In recent years, music source separation has been one of the most intensively studied research areas in music information retrieval. Improvements in deep learning lead to a big progress in music source separation performance. However, most of the previous studies are restricted to separating a few limited number of sources, such as vocals, drums, bass, and other. In this study, we propose a network for audio query-based music source separation that can explicitly encode the source information from a query signal regardless of the number and/or kind of target signals. The proposed method consists of a Query-net and a Separator: given a query and a mixture, the Query-net encodes the query into the latent space, and the Separator estimates masks conditioned by the latent vector, which is then applied to the mixture for separation. The Separator can also generate masks using the latent vector from the training samples, allowing separation in the absence of a query. We evaluate our method on the MUSDB18 dataset, and experimental results show that the proposed method can separate multiple sources with a single network. In addition, through further investigation of the latent space we demonstrate that our method can generate continuous outputs via latent vector interpolation.