# A Four-Stage Data Augmentation Approach to ResNet-Conformer Based Acoustic Modeling for Sound Event Localization and Detection

arXiv: https://arxiv.org/abs/2101.02919 · HTML: https://mlsanity.com/paper/2101.02919/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Jan 2021.

## Built on by (most cited first, 21 of 21)

- [A Survey of Sound Source Localization with Deep Learning Methods](https://mlsanity.com/paper/2109.03465/) (2021)
- [STARSS23: An Audio-Visual Dataset of Spatial Recordings of Real Scenes with Spatiotemporal Annotations of Sound Events](https://mlsanity.com/paper/2306.09126/) (2023)
- [A Real-Time Active Speaker Detection System Integrating an Audio-Visual Signal with a Spatial Querying Mechanism](https://mlsanity.com/paper/2309.08295/) (2023)
- [Stereo Sound Event Localization and Detection with Onscreen/offscreen Classification](https://mlsanity.com/paper/2507.12042/) (2025)
- [DCASE 2021 Task 3: Spectrotemporally-aligned Features for Polyphonic Sound Event Localization and Detection](https://mlsanity.com/paper/2106.15190/) (2021)
- [Enhanced Sound Event Localization and Detection in Real 360-degree audio-visual soundscapes](https://mlsanity.com/paper/2401.17129/) (2024)
- [Leveraging Reverberation and Visual Depth Cues for Sound Event Localization and Detection with Distance Estimation](https://mlsanity.com/paper/2410.22271/) (2024)
- [Exploring Self-Supervised Contrastive Learning of Spatial Sound Event Representation](https://mlsanity.com/paper/2309.15938/) (2023)
- [Locate This, Not That: Class-Conditioned Sound Event DOA Estimation](https://mlsanity.com/paper/2203.04197/) (2022)
- [Learning Multi-Target TDOA Features for Sound Event Localization and Detection](https://mlsanity.com/paper/2408.17166/) (2024)
- [A Review on Sound Source Localization in Robotics: Focusing on Deep Learning Methods](https://mlsanity.com/paper/2507.01143/) (2025)
- [Spatial and Semantic Embedding Integration for Stereo Sound Event Localization and Detection in Regular Videos](https://mlsanity.com/paper/2507.04845/) (2025)
- [Integrating Spatial and Semantic Embeddings for Stereo Sound Event Localization in Videos](https://mlsanity.com/paper/2509.06598/) (2025)
- [Spatial mixup: Directional loudness modification as data augmentation for sound event localization and detection](https://mlsanity.com/paper/2110.06126/) (2021)
- [Inference and Denoise: Causal Inference-based Neural Speech Enhancement](https://mlsanity.com/paper/2211.01189/) (2022)
- [Feature Aggregation in Joint Sound Classification and Localization Neural Networks](https://mlsanity.com/paper/2310.19063/) (2023)
- [Audio-Visual Talker Localization in Video for Spatial Sound Reproduction](https://mlsanity.com/paper/2406.00495/) (2024)
- [DOA-Aware Audio-Visual Self-Supervised Learning for Sound Event Localization and Detection](https://mlsanity.com/paper/2410.22803/) (2024)
- [An Experimental Study on Joint Modeling for Sound Event Localization and Detection with Source Distance Estimation](https://mlsanity.com/paper/2501.10755/) (2025)
- [ToS: A Team of Specialists ensemble framework for Stereo Sound Event Localization and Detection with distance estimation in Video](https://mlsanity.com/paper/2601.17611/) (2026)
- [Supervising Sound Localization by In-the-wild Egomotion](https://mlsanity.com/paper/2610.01388/) (2026)

## Abstract (arXiv)

In this paper, we propose a novel four-stage data augmentation approach to ResNet-Conformer based acoustic modeling for sound event localization and detection (SELD). First, we explore two spatial augmentation techniques, namely audio channel swapping (ACS) and multi-channel simulation (MCS), to deal with data sparsity in SELD. ACS and MDS focus on augmenting the limited training data with expanding direction of arrival (DOA) representations such that the acoustic models trained with the augmented data are robust to localization variations of acoustic sources. Next, time-domain mixing (TDM) and time-frequency masking (TFM) are also investigated to deal with overlapping sound events and data diversity. Finally, ACS, MCS, TDM and TFM are combined in a step-by-step manner to form an effective four-stage data augmentation scheme. Tested on the Detection and Classification of Acoustic Scenes and Events (DCASE) 2020 data set, our proposed augmentation approach greatly improves the system performance, ranking our submitted system in the first place in the SELD task of the DCASE 2020 Challenge. Furthermore, we employ a ResNet-Conformer architecture to model both global and local context dependencies of an audio sequence and win the first place in the DCASE 2022 SELD evaluations.