# SoundDet: Polyphonic Moving Sound Event Detection and Localization from Raw Waveform

arXiv: https://arxiv.org/abs/2106.06969 · HTML: https://mlsanity.com/paper/2106.06969/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 29 references that are in the map.
- First posted on arXiv 13 Jun 2021.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [BSN: Boundary Sensitive Network for Temporal Action Proposal Generation](https://mlsanity.com/paper/1806.02964/) (2018)
- [Speaker Recognition from Raw Waveform with SincNet](https://mlsanity.com/paper/1808.00158/) (2018)
- [Sound Event Localization and Detection of Overlapping Sources Using Convolutional Recurrent Neural Networks](https://mlsanity.com/paper/1807.00129/) (2018)
- [An Improved Event-Independent Network for Polyphonic Sound Event Localization and Detection](https://mlsanity.com/paper/2010.13092/) (2020)
- [Polyphonic Sound Event Detection and Localization using a Two-Stage Strategy](https://mlsanity.com/paper/1905.00268/) (2019)
- [A Sequence Matching Network for Polyphonic Sound Event Localization and Detection](https://mlsanity.com/paper/2002.05865/) (2020)
- [Sound Event Detection in Multichannel Audio Using Spatial and Harmonic Features](https://mlsanity.com/paper/1706.02293/) (2017)
- [A Study of the Complexity and Accuracy of Direction of Arrival Estimation Methods Based on GCC-PHAT for a Pair of Close Microphones](https://mlsanity.com/paper/1811.11787/) (2018)
- [Eventness: Object Detection on Spectrograms for Temporal Localization of Audio Events](https://mlsanity.com/paper/1712.09668/) (2017)
- [Joint Acoustic and Class Inference for Weakly Supervised Sound Event Detection](https://mlsanity.com/paper/1811.04048/) (2018)

## Built on by (most cited first, 8 of 8)

- [A Survey of Sound Source Localization with Deep Learning Methods](https://mlsanity.com/paper/2109.03465/) (2021)
- [STARSS23: An Audio-Visual Dataset of Spatial Recordings of Real Scenes with Spatiotemporal Annotations of Sound Events](https://mlsanity.com/paper/2306.09126/) (2023)
- [RiTTA: Modeling Event Relations in Text-to-Audio Generation](https://mlsanity.com/paper/2412.15922/) (2024)
- [DeepAoANet: Learning Angle of Arrival from Software Defined Radios with Deep Neural Networks](https://mlsanity.com/paper/2112.00695/) (2021)
- [Bayesian inference and neural estimation of acoustic wave propagation](https://mlsanity.com/paper/2305.17749/) (2023)
- [SPEAR: Receiver-to-Receiver Acoustic Neural Warping Field](https://mlsanity.com/paper/2406.11006/) (2024)
- [SoundLoc3D: Invisible 3D Sound Source Localization and Classification Using a Multimodal RGB-D Acoustic Camera](https://mlsanity.com/paper/2412.16861/) (2024)
- [From General-Purpose Audio Tagging to Spatially Grounded Sound Event Localization and Detection](https://mlsanity.com/paper/2606.27751/) (2026)

## Abstract (arXiv)

We present a new framework SoundDet, which is an end-to-end trainable and light-weight framework, for polyphonic moving sound event detection and localization. Prior methods typically approach this problem by preprocessing raw waveform into time-frequency representations, which is more amenable to process with well-established image processing pipelines. Prior methods also detect in segment-wise manner, leading to incomplete and partial detections. SoundDet takes a novel approach and directly consumes the raw, multichannel waveform and treats the spatio-temporal sound event as a complete "sound-object" to be detected. Specifically, SoundDet consists of a backbone neural network and two parallel heads for temporal detection and spatial localization, respectively. Given the large sampling rate of raw waveform, the backbone network first learns a set of phase-sensitive and frequency-selective bank of filters to explicitly retain direction-of-arrival information, whilst being highly computationally and parametrically efficient than standard 1D/2D convolution. A dense sound event proposal map is then constructed to handle the challenges of predicting events with large varying temporal duration. Accompanying the dense proposal map are a temporal overlapness map and a motion smoothness map that measure a proposal's confidence to be an event from temporal detection accuracy and movement consistency perspective. Involving the two maps guarantees SoundDet to be trained in a spatio-temporally unified manner. Experimental results on the public DCASE dataset show the advantage of SoundDet on both segment-based and our newly proposed event-based evaluation system.