# ToS: A Team of Specialists ensemble framework for Stereo Sound Event Localization and Detection with distance estimation in Video

arXiv: https://arxiv.org/abs/2601.17611 · HTML: https://mlsanity.com/paper/2601.17611/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 33 references that are in the map.
- First posted on arXiv 24 Jan 2026.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation](https://mlsanity.com/paper/2211.06687/) (2022)
- [AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Head](https://mlsanity.com/paper/2304.12995/) (2023)
- [Sound Event Localization and Detection of Overlapping Sources Using Convolutional Recurrent Neural Networks](https://mlsanity.com/paper/1807.00129/) (2018)
- [STARSS23: An Audio-Visual Dataset of Spatial Recordings of Real Scenes with Spatiotemporal Annotations of Sound Events](https://mlsanity.com/paper/2306.09126/) (2023)
- [An Improved Event-Independent Network for Polyphonic Sound Event Localization and Detection](https://mlsanity.com/paper/2010.13092/) (2020)
- [A Dataset of Reverberant Spatial Sound Scenes with Moving Sources for Sound Event Localization and Detection](https://mlsanity.com/paper/2006.01919/) (2020)
- [A Four-Stage Data Augmentation Approach to ResNet-Conformer Based Acoustic Modeling for Sound Event Localization and Detection](https://mlsanity.com/paper/2101.02919/) (2021)
- [A Dataset of Dynamic Reverberant Sound Scenes with Directional Interferers for Sound Event Localization and Detection](https://mlsanity.com/paper/2106.06999/) (2021)
- [Sound Event Detection Using Spatial Features and Convolutional Recurrent Neural Network](https://mlsanity.com/paper/1706.02291/) (2017)
- [Sound Event Detection and Localization with Distance Estimation](https://mlsanity.com/paper/2403.11827/) (2024)
- [First Order Ambisonics Domain Spatial Augmentation for DNN-based Direction of Arrival Estimation](https://mlsanity.com/paper/1910.04388/) (2019)
- [Spatial Scaper: A Library to Simulate and Augment Soundscapes for Sound Event Localization and Detection in Realistic Rooms](https://mlsanity.com/paper/2401.12238/) (2024)
- [SALSA-Lite: A Fast and Effective Feature for Polyphonic Sound Event Localization and Detection with Microphone Arrays](https://mlsanity.com/paper/2111.08192/) (2021)
- [SAVGBench: Benchmarking Spatially Aligned Audio-Video Generation](https://mlsanity.com/paper/2412.13462/) (2024)
- [Fusion of Audio and Visual Embeddings for Sound Event Localization and Detection](https://mlsanity.com/paper/2312.09034/) (2023)
- [Two vs. Four-Channel Sound Event Localization and Detection](https://mlsanity.com/paper/2309.13343/) (2023)
- [Generating Diverse Audio-Visual 360 Soundscapes for Sound Event Localization and Detection](https://mlsanity.com/paper/2504.02988/) (2025)
- [Exploring Audio-Visual Information Fusion for Sound Event Localization and Detection In Low-Resource Realistic Scenarios](https://mlsanity.com/paper/2406.15160/) (2024)
- [Stereo Sound Event Localization and Detection with Onscreen/offscreen Classification](https://mlsanity.com/paper/2507.12042/) (2025)
- [Reverberation-based Features for Sound Event Localization and Detection with Distance Estimation](https://mlsanity.com/paper/2504.08644/) (2025)
- [Spatial and Semantic Embedding Integration for Stereo Sound Event Localization and Detection in Regular Videos](https://mlsanity.com/paper/2507.04845/) (2025)
- [Integrating Spatial and Semantic Embeddings for Stereo Sound Event Localization in Videos](https://mlsanity.com/paper/2509.06598/) (2025)

## Abstract (arXiv)

Sound event localization and detection with distance estimation (3D SELD) in video involves identifying active sound events at each time frame while estimating their spatial coordinates. This multimodal task requires joint reasoning across semantic, spatial, and temporal dimensions, a challenge that single models often struggle to address effectively. To tackle this, we introduce the Team of Specialists (ToS) ensemble framework, which integrates three complementary sub-networks: a spatio-linguistic model, a spatio-temporal model, and a tempo-linguistic model. Each sub-network specializes in a unique pair of dimensions, contributing distinct insights to the final prediction, akin to a collaborative team with diverse expertise. ToS has been benchmarked against state-of-the-art audio-visual models for 3D SELD on the DCASE2025 Task 3 Stereo SELD development set, consistently outperforming existing methods across key metrics. Future work will extend this proof of concept by strengthening the specialists with appropriate tasks, training, and pre-training curricula.