# The Cone of Silence: Speech Separation by Localization

arXiv: https://arxiv.org/abs/2010.06007 · HTML: https://mlsanity.com/paper/2010.06007/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Oct 2020.

## Built on by (most cited first, 21 of 21)

- [SoundSpaces 2.0: A Simulation Platform for Visual-Acoustic Learning](https://mlsanity.com/paper/2206.08312/) (2022)
- [Music Source Separation with Band-split RNN](https://mlsanity.com/paper/2209.15174/) (2022)
- [Contrastive Positive Sample Propagation along the Audio-Visual Event Line](https://mlsanity.com/paper/2211.09980/) (2022)
- [A Survey of Sound Source Localization with Deep Learning Methods](https://mlsanity.com/paper/2109.03465/) (2021)
- [MESH2IR: Neural Acoustic Impulse Response Generator for Complex 3D Scenes](https://mlsanity.com/paper/2205.09248/) (2022)
- [Look Once to Hear: Target Speech Hearing with Noisy Examples](https://mlsanity.com/paper/2405.06289/) (2024)
- [Multi-channel Speech Separation Using Spatially Selective Deep Non-linear Filters](https://mlsanity.com/paper/2304.12023/) (2023)
- [ReZero: Region-customizable Sound Extraction](https://mlsanity.com/paper/2308.16892/) (2023)
- [Towards Improved Room Impulse Response Estimation for Speech Recognition](https://mlsanity.com/paper/2211.04473/) (2022)
- [Hybrid Neural Networks for On-device Directional Hearing](https://mlsanity.com/paper/2112.05893/) (2021)
- [Sim2Real Transfer for Audio-Visual Navigation with Frequency-Adaptive Acoustic Field Prediction](https://mlsanity.com/paper/2405.02821/) (2024)
- [Implicit Neural Spatial Filtering for Multichannel Source Separation in the Waveform Domain](https://mlsanity.com/paper/2206.15423/) (2022)
- [Points2Sound: From mono to binaural audio using 3D point cloud scenes](https://mlsanity.com/paper/2104.12462/) (2021)
- [Learning to Separate Voices by Spatial Regions](https://mlsanity.com/paper/2207.04203/) (2022)
- [Spatially Selective Deep Non-linear Filters for Speaker Extraction](https://mlsanity.com/paper/2211.02420/) (2022)
- [Multi-Microphone Speaker Separation by Spatial Regions](https://mlsanity.com/paper/2303.07143/) (2023)
- [UNSSOR: Unsupervised Neural Speech Separation by Leveraging Over-determined Training Mixtures](https://mlsanity.com/paper/2305.20054/) (2023)
- [Spatial Speech Translation: Translating Across Space With Binaural Hearables](https://mlsanity.com/paper/2504.18715/) (2025)
- [SonicSieve: Bringing Directional Speech Extraction to Smartphones Using Acoustic Microstructures](https://mlsanity.com/paper/2504.10793/) (2025)
- [A Review on Sound Source Localization in Robotics: Focusing on Deep Learning Methods](https://mlsanity.com/paper/2507.01143/) (2025)
- [Addressing Feature Imbalance in Sound Source Separation](https://mlsanity.com/paper/2309.05287/) (2023)

## Abstract (arXiv)

Given a multi-microphone recording of an unknown number of speakers talking concurrently, we simultaneously localize the sources and separate the individual speakers. At the core of our method is a deep network, in the waveform domain, which isolates sources within an angular region θ ± w/2, given an angle of interest θ and angular window size w. By exponentially decreasing w, we can perform a binary search to localize and separate all sources in logarithmic time. Our algorithm allows for an arbitrary number of potentially moving speakers at test time, including more speakers than seen during training. Experiments demonstrate state-of-the-art performance for both source separation and source localization, particularly in high levels of background noise.