# Class-conditional embeddings for music source separation

arXiv: https://arxiv.org/abs/1811.03076 · HTML: https://mlsanity.com/paper/1811.03076/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Nov 2018.

## Built on by (most cited first, 12 of 12)

- [Move2Hear: Active Audio-Visual Source Separation](https://mlsanity.com/paper/2105.07142/) (2021)
- [Weakly-Supervised Audio-Visual Segmentation](https://mlsanity.com/paper/2311.15080/) (2023)
- [A Unified Model for Zero-shot Music Source Separation, Transcription and Synthesis](https://mlsanity.com/paper/2108.03456/) (2021)
- [Heterogeneous Target Speech Separation](https://mlsanity.com/paper/2204.03594/) (2022)
- [A Stem-Agnostic Single-Decoder System for Music Source Separation Beyond Four Stems](https://mlsanity.com/paper/2406.18747/) (2024)
- [Optimal Condition Training for Target Source Separation](https://mlsanity.com/paper/2211.05927/) (2022)
- [Semantic Grouping Network for Audio Source Separation](https://mlsanity.com/paper/2407.03736/) (2024)
- [Locate This, Not That: Class-Conditioned Sound Event DOA Estimation](https://mlsanity.com/paper/2203.04197/) (2022)
- [LightSAFT: Lightweight Latent Source Aware Frequency Transform for Source Separation](https://mlsanity.com/paper/2111.12516/) (2021)
- [SANE-TTS: Stable And Natural End-to-End Multilingual Text-to-Speech](https://mlsanity.com/paper/2206.12132/) (2022)
- [Pre-trained Spatial Priors on Multichannel NMF for Music Source Separation](https://mlsanity.com/paper/2310.05821/) (2023)
- [A Conditioned UNet for Music Source Separation](https://mlsanity.com/paper/2512.15532/) (2025)

## Abstract (arXiv)

Isolating individual instruments in a musical mixture has a myriad of potential applications, and seems imminently achievable given the levels of performance reached by recent deep learning methods. While most musical source separation techniques learn an independent model for each instrument, we propose using a common embedding space for the time-frequency bins of all instruments in a mixture inspired by deep clustering and deep attractor networks. Additionally, an auxiliary network is used to generate parameters of a Gaussian mixture model (GMM) where the posterior distribution over GMM components in the embedding space can be used to create a mask that separates individual sources from a mixture. In addition to outperforming a mask-inference baseline on the MUSDB-18 dataset, our embedding space is easily interpretable and can be used for query-based separation.