mlsanityData to 7 Oct 2026

ToS: A Team of Specialists ensemble framework for Stereo Sound Event Localization and Detection with distance estimation in Video

arXiv 2601.17611 ↗First posted 24 Jan 2026eess.AS cs.LG cs.SDcited by 0

Not on the map. Why?

Abstract (arXiv)

Sound event localization and detection with distance estimation (3D SELD) in video involves identifying active sound events at each time frame while estimating their spatial coordinates. This multimodal task requires joint reasoning across semantic, spatial, and temporal dimensions, a challenge that single models often struggle to address effectively. To tackle this, we introduce the Team of Specialists (ToS) ensemble framework, which integrates three complementary sub-networks: a spatio-linguistic model, a spatio-temporal model, and a tempo-linguistic model. Each sub-network specializes in a unique pair of dimensions, contributing distinct insights to the final prediction, akin to a collaborative team with diverse expertise. ToS has been benchmarked against state-of-the-art audio-visual models for 3D SELD on the DCASE2025 Task 3 Stereo SELD development set, consistently outperforming existing methods across key metrics. Future work will extend this proof of concept by strengthening the specialists with appropriate tasks, training, and pre-training curricula.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 26 of its 33 references that are in the map. First posted on arXiv 24 Jan 2026.

Builds on

26 of 33 references.

References · 26

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  5. 2022Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentationcited by 707
  6. 2023AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Headcited by 264
  7. 2018Sound Event Localization and Detection of Overlapping Sources Using Convolutional Recurrent Neural Networkscited by 84
  8. 2023STARSS23: An Audio-Visual Dataset of Spatial Recordings of Real Scenes with Spatiotemporal Annotations of Sound Eventscited by 39
  9. 2020An Improved Event-Independent Network for Polyphonic Sound Event Localization and Detectioncited by 25
  10. 2020A Dataset of Reverberant Spatial Sound Scenes with Moving Sources for Sound Event Localization and Detectioncited by 21
  11. 2021A Four-Stage Data Augmentation Approach to ResNet-Conformer Based Acoustic Modeling for Sound Event Localization and Detectioncited by 21
  12. 2021A Dataset of Dynamic Reverberant Sound Scenes with Directional Interferers for Sound Event Localization and Detectioncited by 20
14 more
  1. 2017Sound Event Detection Using Spatial Features and Convolutional Recurrent Neural Networkcited by 14
  2. 2024Sound Event Detection and Localization with Distance Estimationcited by 13
  3. 2019First Order Ambisonics Domain Spatial Augmentation for DNN-based Direction of Arrival Estimationcited by 12
  4. 2024Spatial Scaper: A Library to Simulate and Augment Soundscapes for Sound Event Localization and Detection in Realistic Roomscited by 12
  5. 2021SALSA-Lite: A Fast and Effective Feature for Polyphonic Sound Event Localization and Detection with Microphone Arrayscited by 10
  6. 2024SAVGBench: Benchmarking Spatially Aligned Audio-Video Generationcited by 10
  7. 2023Fusion of Audio and Visual Embeddings for Sound Event Localization and Detectioncited by 9
  8. 2023Two vs. Four-Channel Sound Event Localization and Detectioncited by 6
  9. 2025Generating Diverse Audio-Visual 360 Soundscapes for Sound Event Localization and Detectioncited by 6
  10. 2024Exploring Audio-Visual Information Fusion for Sound Event Localization and Detection In Low-Resource Realistic Scenarioscited by 4
  11. 2025Stereo Sound Event Localization and Detection with Onscreen/offscreen Classificationcited by 4
  12. 2025Reverberation-based Features for Sound Event Localization and Detection with Distance Estimationcited by 3
  13. 2025Spatial and Semantic Embedding Integration for Stereo Sound Event Localization and Detection in Regular Videoscited by 1
  14. 2025Integrating Spatial and Semantic Embeddings for Stereo Sound Event Localization in Videoscited by 1

Built on by

Not cited yet.