# Learning to Separate Voices by Spatial Regions

arXiv: https://arxiv.org/abs/2207.04203 · HTML: https://mlsanity.com/paper/2207.04203/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 36 references that are in the map.
- First posted on arXiv 9 Jul 2022.

## Builds on (most cited first, 24 of 24)

- [SDR - half-baked or well done?](https://mlsanity.com/paper/1811.02508/) (2018)
- [Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separation](https://mlsanity.com/paper/1809.07454/) (2018)
- [Deep clustering: Discriminative embeddings for segmentation and separation](https://mlsanity.com/paper/1508.04306/) (2015)
- [Permutation Invariant Training of Deep Models for Speaker-Independent Multi-talker Speech Separation](https://mlsanity.com/paper/1607.00325/) (2016)
- [Attention is All You Need in Speech Separation](https://mlsanity.com/paper/2010.13154/) (2020)
- [Dual-path RNN: efficient long sequence modeling for time-domain single-channel speech separation](https://mlsanity.com/paper/1910.06379/) (2019)
- [LibriMix: An Open-Source Dataset for Generalizable Speech Separation](https://mlsanity.com/paper/2005.11262/) (2020)
- [Voice Separation with an Unknown Number of Multiple Speakers](https://mlsanity.com/paper/2003.01531/) (2020)
- [Deep attractor network for single-microphone speaker separation](https://mlsanity.com/paper/1611.08930/) (2016)
- [FaSNet: Low-latency Adaptive Beamforming for Multi-microphone Audio Processing](https://mlsanity.com/paper/1909.13387/) (2019)
- [What's All the FUSS About Free Universal Sound Separation Data?](https://mlsanity.com/paper/2011.00803/) (2020)
- [The Cone of Silence: Speech Separation by Localization](https://mlsanity.com/paper/2010.06007/) (2020)
- [Speaker-independent Speech Separation with Deep Attractor Network](https://mlsanity.com/paper/1707.03634/) (2017)
- [Unsupervised training of a deep clustering model for multichannel blind source separation](https://mlsanity.com/paper/1904.01340/) (2019)
- [Recursive speech separation for unknown number of speakers](https://mlsanity.com/paper/1904.03065/) (2019)
- [Unsupervised Deep Clustering for Source Separation: Direct Learning from Mixtures using Spatial Information](https://mlsanity.com/paper/1811.01531/) (2018)
- [Enhancing End-to-End Multi-channel Speech Separation via Spatial Feature Learning](https://mlsanity.com/paper/2003.03927/) (2020)
- [End-to-End Multi-Channel Speech Separation](https://mlsanity.com/paper/1905.06286/) (2019)
- [Real-time binaural speech separation with preserved spatial cues](https://mlsanity.com/paper/2002.06637/) (2020)
- [Many-Speakers Single Channel Speech Separation with Optimal Permutation Training](https://mlsanity.com/paper/2104.08955/) (2021)
- [Single channel voice separation for unknown number of speakers under reverberant and noisy settings](https://mlsanity.com/paper/2011.02329/) (2020)
- [Bootstrapping single-channel source separation via unsupervised spatial clustering on stereo mixtures](https://mlsanity.com/paper/1811.02130/) (2018)
- [SAGRNN: Self-Attentive Gated RNN for Binaural Speaker Separation with Interaural Cue Preservation](https://mlsanity.com/paper/2009.01381/) (2020)
- [Building Corpora for Single-Channel Speech Separation Across Multiple Domains](https://mlsanity.com/paper/1811.02641/) (2018)

## Built on by (most cited first, 4 of 4)

- [ReZero: Region-customizable Sound Extraction](https://mlsanity.com/paper/2308.16892/) (2023)
- [ArrayDPS: Unsupervised Blind Speech Separation with a Diffusion Prior](https://mlsanity.com/paper/2505.05657/) (2025)
- [Multi-Microphone Speaker Separation by Spatial Regions](https://mlsanity.com/paper/2303.07143/) (2023)
- [Spatial Speech Translation: Translating Across Space With Binaural Hearables](https://mlsanity.com/paper/2504.18715/) (2025)

## Abstract (arXiv)

We consider the problem of audio voice separation for binaural applications, such as earphones and hearing aids. While today's neural networks perform remarkably well (separating 4+ sources with 2 microphones) they assume a known or fixed maximum number of sources, K. Moreover, today's models are trained in a supervised manner, using training data synthesized from generic sources, environments, and human head shapes. This paper intends to relax both these constraints at the expense of a slight alteration in the problem definition. We observe that, when a received mixture contains too many sources, it is still helpful to separate them by region, i.e., isolating signal mixtures from each conical sector around the user's head. This requires learning the fine-grained spatial properties of each region, including the signal distortions imposed by a person's head. We propose a two-stage self-supervised framework in which overheard voices from earphones are pre-processed to extract relatively clean personalized signals, which are then used to train a region-wise separation model. Results show promising performance, underscoring the importance of personalization over a generic supervised approach. (audio samples available at our project website: https://uiuc-earable-computing.github.io/binaural/. We believe this result could help real-world applications in selective hearing, noise cancellation, and audio augmented reality.