# Deep attractor network for single-microphone speaker separation

arXiv: https://arxiv.org/abs/1611.08930 · HTML: https://mlsanity.com/paper/1611.08930/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Nov 2016.

## Built on by (most cited first, 26 of 26)

- [An efficient encoder-decoder architecture with top-down attention for speech separation](https://mlsanity.com/paper/2209.15200/) (2022)
- [Looking into Your Speech: Learning Cross-modal Affinity for Audio-visual Speech Separation](https://mlsanity.com/paper/2104.02775/) (2021)
- [Personalized Speech Enhancement: New Models and Comprehensive Evaluation](https://mlsanity.com/paper/2110.09625/) (2021)
- [Towards Neural Diarization for Unlimited Numbers of Speakers Using Global and Local Attractors](https://mlsanity.com/paper/2107.01545/) (2021)
- [Many-Speakers Single Channel Speech Separation with Optimal Permutation Training](https://mlsanity.com/paper/2104.08955/) (2021)
- [Online Neural Diarization of Unlimited Numbers of Speakers Using Global and Local Attractors](https://mlsanity.com/paper/2206.02432/) (2022)
- [ESPnet-SE++: Speech Enhancement for Robust Speech Recognition, Translation, and Understanding](https://mlsanity.com/paper/2207.09514/) (2022)
- [MIMO Self-attentive RNN Beamformer for Multi-speaker Speech Separation](https://mlsanity.com/paper/2104.08450/) (2021)
- [A Comparative Study on Speaker-attributed Automatic Speech Recognition in Multi-party Meetings](https://mlsanity.com/paper/2203.16834/) (2022)
- [AudioSlots: A slot-centric generative model for audio separation](https://mlsanity.com/paper/2305.05591/) (2023)
- [Learning to Separate Voices by Spatial Regions](https://mlsanity.com/paper/2207.04203/) (2022)
- [Deep neural network techniques for monaural speech enhancement: state of the art analysis](https://mlsanity.com/paper/2212.00369/) (2022)
- [Multiple Choice Learning for Efficient Speech Separation with Many Speakers](https://mlsanity.com/paper/2411.18497/) (2024)
- [Tune-In: Training Under Negative Environments with Interference for Attention Networks Simulating Cocktail Party Effect](https://mlsanity.com/paper/2103.01461/) (2021)
- [Plug-and-Play Co-Occurring Face Attention for Robust Audio-Visual Speaker Extraction](https://mlsanity.com/paper/2505.20635/) (2025)
- [A Data-Driven Prism: Multi-View Source Separation with Diffusion Model Priors](https://mlsanity.com/paper/2510.05205/) (2025)
- [Single-channel speech separation using Soft-minimum Permutation Invariant Training](https://mlsanity.com/paper/2111.08635/) (2021)
- [Blind Source Separation of Single-Channel Mixtures via Multi-Encoder Autoencoders](https://mlsanity.com/paper/2309.07138/) (2023)
- [Unsupervised Speaker Diarization in Distributed IoT Networks Using Federated Learning](https://mlsanity.com/paper/2404.10842/) (2024)
- [Blind Speech Separation and Dereverberation using Neural Beamforming](https://mlsanity.com/paper/2103.13443/) (2021)
- [Manifold-Aware Deep Clustering: Maximizing Angles between Embedding Vectors Based on Regular Simplex](https://mlsanity.com/paper/2106.02331/) (2021)
- [Online Self-Attentive Gated RNNs for Real-Time Speaker Separation](https://mlsanity.com/paper/2106.13493/) (2021)
- [An Empirical Analysis on the Vulnerabilities of End-to-End Speech Segregation Models](https://mlsanity.com/paper/2206.09556/) (2022)
- [Individualized Conditioning and Negative Distances for Speaker Separation](https://mlsanity.com/paper/2210.06368/) (2022)
- [Improving Neural Diarization through Speaker Attribute Attractors and Local Dependency Modeling](https://mlsanity.com/paper/2506.05593/) (2025)
- [TGCM: Topic-Guided Consistency Modeling for One-Step Disentanglement of Interleaved APT Technique Sequences](https://mlsanity.com/paper/2606.18651/) (2026)

## Abstract (arXiv)

Despite the overwhelming success of deep learning in various speech processing tasks, the problem of separating simultaneous speakers in a mixture remains challenging. Two major difficulties in such systems are the arbitrary source permutation and unknown number of sources in the mixture. We propose a novel deep learning framework for single channel speech separation by creating attractor points in high dimensional embedding space of the acoustic signals which pull together the time-frequency bins corresponding to each source. Attractor points in this study are created by finding the centroids of the sources in the embedding space, which are subsequently used to determine the similarity of each bin in the mixture to each source. The network is then trained to minimize the reconstruction error of each source by optimizing the embeddings. The proposed model is different from prior works in that it implements an end-to-end training, and it does not depend on the number of sources in the mixture. Two strategies are explored in the test time, K-means and fixed attractor points, where the latter requires no post-processing and can be implemented in real-time. We evaluated our system on Wall Street Journal dataset and show 5.49% improvement over the previous state-of-the-art methods.