# Dual input neural networks for positional sound source localization

arXiv: https://arxiv.org/abs/2308.04169 · HTML: https://mlsanity.com/paper/2308.04169/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 61 references that are in the map.
- First posted on arXiv 8 Aug 2023.

## Builds on (most cited first, 19 of 19)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Multimodal Machine Learning: A Survey and Taxonomy](https://mlsanity.com/paper/1705.09406/) (2017)
- [Permutation Invariant Training of Deep Models for Speaker-Independent Multi-talker Speech Separation](https://mlsanity.com/paper/1607.00325/) (2016)
- [Pyroomacoustics: A Python package for audio room simulations and array processing algorithms](https://mlsanity.com/paper/1710.04196/) (2017)
- [Convolutional Recurrent Neural Networks for Music Classification](https://mlsanity.com/paper/1609.04243/) (2016)
- [A Survey of Sound Source Localization with Deep Learning Methods](https://mlsanity.com/paper/2109.03465/) (2021)
- [Parameterized Machine Learning for High-Energy Physics](https://mlsanity.com/paper/1601.07913/) (2016)
- [Direction of arrival estimation for multiple sound sources using convolutional recurrent neural network](https://mlsanity.com/paper/1710.10059/) (2017)
- [Deep Neural Networks for Multiple Speaker Detection and Localization](https://mlsanity.com/paper/1711.11565/) (2017)
- [Polyphonic Sound Event Detection and Localization using a Two-Stage Strategy](https://mlsanity.com/paper/1905.00268/) (2019)
- [Broadband DOA estimation using Convolutional neural networks trained with noise signals](https://mlsanity.com/paper/1705.00919/) (2017)
- [Towards End-to-End Acoustic Localization using Deep Learning: from Audio Signal to Source Position Coordinates](https://mlsanity.com/paper/1807.11094/) (2018)
- [L3DAS22 Challenge: Learning 3D Audio Sources in a Real Office Environment](https://mlsanity.com/paper/2202.10372/) (2022)
- [Libri-adhoc40: A dataset collected from synchronized ad-hoc microphone arrays](https://mlsanity.com/paper/2103.15118/) (2021)
- [A Universal Deep Room Acoustics Estimator](https://mlsanity.com/paper/2109.14436/) (2021)
- [Evaluation of an open-source implementation of the SRP-PHAT algorithm within the 2018 LOCATA challenge](https://mlsanity.com/paper/1812.05901/) (2018)

## Built on by (most cited first, 3 of 3)

- [wav2pos: Sound Source Localization using Masked Autoencoders](https://mlsanity.com/paper/2408.15771/) (2024)
- [SONNET: Enhancing Time Delay Estimation by Leveraging Simulated Audio](https://mlsanity.com/paper/2411.13179/) (2024)
- [Efficient and Microphone-Fault-Tolerant 3D Sound Source Localization](https://mlsanity.com/paper/2505.20961/) (2025)

## Abstract (arXiv)

In many signal processing applications, metadata may be advantageously used in conjunction with a high dimensional signal to produce a desired output. In the case of classical Sound Source Localization (SSL) algorithms, information from a high dimensional, multichannel audio signals received by many distributed microphones is combined with information describing acoustic properties of the scene, such as the microphones' coordinates in space, to estimate the position of a sound source. We introduce Dual Input Neural Networks (DI-NNs) as a simple and effective way to model these two data types in a neural network. We train and evaluate our proposed DI-NN on scenarios of varying difficulty and realism and compare it against an alternative architecture, a classical Least-Squares (LS) method as well as a classical Convolutional Recurrent Neural Network (CRNN). Our results show that the DI-NN significantly outperforms the baselines, achieving a five times lower localization error than the LS method and two times lower than the CRNN in a test dataset of real recordings.