# Deep Clustering and Conventional Networks for Music Separation: Stronger Together

arXiv: https://arxiv.org/abs/1611.06265 · HTML: https://mlsanity.com/paper/1611.06265/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Nov 2016.

## Built on by (most cited first, 10 of 10)

- [Music Source Separation with Band-split RNN](https://mlsanity.com/paper/2209.15174/) (2022)
- [Music Separation Enhancement with Generative Modeling](https://mlsanity.com/paper/2208.12387/) (2022)
- [Audiovisual Singing Voice Separation](https://mlsanity.com/paper/2107.00231/) (2021)
- [Deep neural network techniques for monaural speech enhancement: state of the art analysis](https://mlsanity.com/paper/2212.00369/) (2022)
- [Single-channel speech separation using Soft-minimum Permutation Invariant Training](https://mlsanity.com/paper/2111.08635/) (2021)
- [Robustness of Speech Separation Models for Similar-pitch Speakers](https://mlsanity.com/paper/2407.15749/) (2024)
- [MARS-Sep: Multimodal-Aligned Reinforced Sound Separation](https://mlsanity.com/paper/2510.10509/) (2025)
- [Manifold-Aware Deep Clustering: Maximizing Angles between Embedding Vectors Based on Regular Simplex](https://mlsanity.com/paper/2106.02331/) (2021)
- [LightSAFT: Lightweight Latent Source Aware Frequency Transform for Source Separation](https://mlsanity.com/paper/2111.12516/) (2021)
- [Coarse-to-Fine Recursive Speech Separation for Unknown Number of Speakers](https://mlsanity.com/paper/2203.16054/) (2022)

## Abstract (arXiv)

Deep clustering is the first method to handle general audio separation scenarios with multiple sources of the same type and an arbitrary number of sources, performing impressively in speaker-independent speech separation tasks. However, little is known about its effectiveness in other challenging situations such as music source separation. Contrary to conventional networks that directly estimate the source signals, deep clustering generates an embedding for each time-frequency bin, and separates sources by clustering the bins in the embedding space. We show that deep clustering outperforms conventional networks on a singing voice separation task, in both matched and mismatched conditions, even though conventional networks have the advantage of end-to-end training for best signal approximation, presumably because its more flexible objective engenders better regularization. Since the strengths of deep clustering and conventional network architectures appear complementary, we explore combining them in a single hybrid network trained via an approach akin to multi-task learning. Remarkably, the combination significantly outperforms either of its components.