# Deep Neural Networks for Multiple Speaker Detection and Localization

arXiv: https://arxiv.org/abs/1711.11565 · HTML: https://mlsanity.com/paper/1711.11565/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Nov 2017.

## Built on by (most cited first, 23 of 23)

- [A Survey of Sound Source Localization with Deep Learning Methods](https://mlsanity.com/paper/2109.03465/) (2021)
- [Lightweight Neural Architecture Search for Temporal Convolutional Networks at the Edge](https://mlsanity.com/paper/2301.10281/) (2023)
- [PILOT: Introducing Transformers for Probabilistic Sound Event Localization](https://mlsanity.com/paper/2106.03903/) (2021)
- [Extending GCC-PHAT using Shift Equivariant Neural Networks](https://mlsanity.com/paper/2208.04654/) (2022)
- [Exploiting Attention-based Sequence-to-Sequence Architectures for Sound Event Localization](https://mlsanity.com/paper/2103.00417/) (2021)
- [A Deep Reinforcement Learning Approach to Audio-Based Navigation in a Multi-Speaker Environment](https://mlsanity.com/paper/2105.04488/) (2021)
- [Multi-goal Audio-visual Navigation using Sound Direction Map](https://mlsanity.com/paper/2308.00219/) (2023)
- [LocSelect: Target Speaker Localization with an Auditory Selective Hearing Mechanism](https://mlsanity.com/paper/2310.10497/) (2023)
- [Pruning In Time (PIT): A Lightweight Network Architecture Optimizer for Temporal Convolutional Networks](https://mlsanity.com/paper/2203.14768/) (2022)
- [MIMO-DoAnet: Multi-channel Input and Multiple Outputs DoA Network with Unknown Number of Sound Sources](https://mlsanity.com/paper/2207.07307/) (2022)
- [Dual input neural networks for positional sound source localization](https://mlsanity.com/paper/2308.04169/) (2023)
- [Audio-video fusion strategies for active speaker detection in meetings](https://mlsanity.com/paper/2206.10411/) (2022)
- [A Deep Reinforcement Learning Approach for Audio-based Navigation and Audio Source Localization in Multi-speaker Environments](https://mlsanity.com/paper/2110.12778/) (2021)
- [Iterative Sound Source Localization for Unknown Number of Sources](https://mlsanity.com/paper/2206.12273/) (2022)
- [A Teacher-Student approach for extracting informative speaker embeddings from speech mixtures](https://mlsanity.com/paper/2306.00634/) (2023)
- [Leveraging Visual Supervision for Array-based Active Speaker Detection and Localization](https://mlsanity.com/paper/2312.14021/) (2023)
- [A Review on Sound Source Localization in Robotics: Focusing on Deep Learning Methods](https://mlsanity.com/paper/2507.01143/) (2025)
- [MIMO-DBnet: Multi-channel Input and Multiple Outputs DOA-aware Beamforming Network for Speech Separation](https://mlsanity.com/paper/2212.03401/) (2022)
- [Multi-microphone Automatic Speech Segmentation in Meetings Based on Circular Harmonics Features](https://mlsanity.com/paper/2306.04268/) (2023)
- [Mask-Weighted Spatial Likelihood Coding for Speaker-Independent Joint Localization and Mask Estimation](https://mlsanity.com/paper/2410.19595/) (2024)
- [Analytic Incremental Learning For Sound Source Localization With Imbalance Rectification](https://mlsanity.com/paper/2601.18335/) (2026)
- [NeuralMUSIC: A Hybrid Neural-Subspace Framework for Robot Sound Source Localization](https://mlsanity.com/paper/2606.18664/) (2026)
- [CNN-Transformer with Sigmoid Label for High-Resolution Sub-Degree DOA Estimation](https://mlsanity.com/paper/2610.13433/) (2026)

## Abstract (arXiv)

We propose to use neural networks for simultaneous detection and localization of multiple sound sources in human-robot interaction. In contrast to conventional signal processing techniques, neural network-based sound source localization methods require fewer strong assumptions about the environment. Previous neural network-based methods have been focusing on localizing a single sound source, which do not extend to multiple sources in terms of detection and localization. In this paper, we thus propose a likelihood-based encoding of the network output, which naturally allows the detection of an arbitrary number of sources. In addition, we investigate the use of sub-band cross-correlation information as features for better localization in sound mixtures, as well as three different network architectures based on different motivations. Experiments on real data recorded from a robot show that our proposed methods significantly outperform the popular spatial spectrum-based approaches.