# Sound Event Localization and Detection of Overlapping Sources Using Convolutional Recurrent Neural Networks

arXiv: https://arxiv.org/abs/1807.00129 · HTML: https://mlsanity.com/paper/1807.00129/

## Key facts

- Cited by 85 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Jun 2018.

## Built on by (most cited first, 30 of 85)

- [A Survey of Sound Source Localization with Deep Learning Methods](https://mlsanity.com/paper/2109.03465/) (2021)
- [STARSS23: An Audio-Visual Dataset of Spatial Recordings of Real Scenes with Spatiotemporal Annotations of Sound Events](https://mlsanity.com/paper/2306.09126/) (2023)
- [BAT: Learning to Reason about Spatial Sounds with Large Language Models](https://mlsanity.com/paper/2402.01591/) (2024)
- [PHNNs: Lightweight Neural Networks via Parameterized Hypercomplex Convolutions](https://mlsanity.com/paper/2110.04176/) (2021)
- [Visual Sound Localization in the Wild by Cross-Modal Interference Erasing](https://mlsanity.com/paper/2202.06406/) (2022)
- [Learning Spatially-Aware Language and Audio Embeddings](https://mlsanity.com/paper/2409.11369/) (2024)
- [SAVVY: Spatial Awareness via Audio-Visual LLMs through Seeing and Hearing](https://mlsanity.com/paper/2506.05414/) (2025)
- [Sound Event Detection and Localization with Distance Estimation](https://mlsanity.com/paper/2403.11827/) (2024)
- [Resilient Multiple Choice Learning: A learned scoring scheme with application to audio scene analysis](https://mlsanity.com/paper/2311.01052/) (2023)
- [ImmerseDiffusion: A Generative Spatial Audio Latent Diffusion Model](https://mlsanity.com/paper/2410.14945/) (2024)
- [L3DAS22 Challenge: Learning 3D Audio Sources in a Real Office Environment](https://mlsanity.com/paper/2202.10372/) (2022)
- [L3DAS21 Challenge: Machine Learning for 3D Audio Signal Processing](https://mlsanity.com/paper/2104.05499/) (2021)
- [Panoramic Video Salient Object Detection with Ambisonic Audio Guidance](https://mlsanity.com/paper/2211.14419/) (2022)
- [SoundDet: Polyphonic Moving Sound Event Detection and Localization from Raw Waveform](https://mlsanity.com/paper/2106.06969/) (2021)
- [Self-Supervised Moving Vehicle Detection from Audio-Visual Cues](https://mlsanity.com/paper/2201.12771/) (2022)
- [Dual Quaternion Ambisonics Array for Six-Degree-of-Freedom Acoustic Representation](https://mlsanity.com/paper/2204.01851/) (2022)
- [PILOT: Introducing Transformers for Probabilistic Sound Event Localization](https://mlsanity.com/paper/2106.03903/) (2021)
- [Spatial LibriSpeech: An Augmented Dataset for Spatial Audio Learning](https://mlsanity.com/paper/2308.09514/) (2023)
- [Exploiting Attention-based Sequence-to-Sequence Architectures for Sound Event Localization](https://mlsanity.com/paper/2103.00417/) (2021)
- [Winner-takes-all learners are geometry-aware conditional density estimators](https://mlsanity.com/paper/2406.04706/) (2024)
- [SELD-Mamba: Selective State-Space Model for Sound Event Localization and Detection with Source Distance Estimation](https://mlsanity.com/paper/2408.05057/) (2024)
- [Diff-SAGe: End-to-End Spatial Audio Generation Using Diffusion Models](https://mlsanity.com/paper/2410.11299/) (2024)
- [PhaseCoder: Microphone Geometry-Agnostic Spatial Audio Understanding for Multimodal LLMs](https://mlsanity.com/paper/2601.21124/) (2026)
- [Semi-supervised source localization in reverberant environments with deep generative modeling](https://mlsanity.com/paper/2101.10636/) (2021)
- [Guided Masked Self-Distillation Modeling for Distributed Multimedia Sensor Event Analysis](https://mlsanity.com/paper/2404.08264/) (2024)
- [TF-Mamba: A Time-Frequency Network for Sound Source Localization](https://mlsanity.com/paper/2409.05034/) (2024)
- [Sci-Phi: A Large Language Model Spatial Audio Descriptor](https://mlsanity.com/paper/2510.05542/) (2025)
- [Not in Sync: Unveiling Temporal Bias in Audio Chat Models](https://mlsanity.com/paper/2510.12185/) (2025)
- [Decompose the Sounds and Pixels, Recompose the Events](https://mlsanity.com/paper/2112.11547/) (2021)
- [Robust detection of overlapping bioacoustic sound events](https://mlsanity.com/paper/2503.02389/) (2025)

## Abstract (arXiv)

In this paper, we propose a convolutional recurrent neural network for joint sound event localization and detection (SELD) of multiple overlapping sound events in three-dimensional (3D) space. The proposed network takes a sequence of consecutive spectrogram time-frames as input and maps it to two outputs in parallel. As the first output, the sound event detection (SED) is performed as a multi-label classification task on each time-frame producing temporal activity for all the sound event classes. As the second output, localization is performed by estimating the 3D Cartesian coordinates of the direction-of-arrival (DOA) for each sound event class using multi-output regression. The proposed method is able to associate multiple DOAs with respective sound event labels and further track this association with respect to time. The proposed method uses separately the phase and magnitude component of the spectrogram calculated on each audio channel as the feature, thereby avoiding any method- and array-specific feature extraction. The method is evaluated on five Ambisonic and two circular array format datasets with different overlapping sound events in anechoic, reverberant and real-life scenarios. The proposed method is compared with two SED, three DOA estimation, and one SELD baselines. The results show that the proposed method is generic and applicable to any array structures, robust to unseen DOA values, reverberation, and low SNR scenarios. The proposed method achieved a consistently higher recall of the estimated number of DOAs across datasets in comparison to the best baseline. Additionally, this recall was observed to be significantly better than the best baseline method for a higher number of overlapping sound events.