# Differentiable Tracking-Based Training of Deep Learning Sound Source Localizers

arXiv: https://arxiv.org/abs/2111.00030 · HTML: https://mlsanity.com/paper/2111.00030/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Oct 2021.

## Built on by (most cited first, 8 of 8)

- [Direction of Arrival Estimation of Sound Sources Using Icosahedral CNNs](https://mlsanity.com/paper/2203.16940/) (2022)
- [Position tracking of a varying number of sound sources with sliding permutation invariant training](https://mlsanity.com/paper/2210.14536/) (2022)
- [Speaker Embeddings to Improve Tracking of Intermittent and Moving Speakers](https://mlsanity.com/paper/2506.19875/) (2025)
- [SIRUP: A diffusion-based virtual upmixer of steering vectors for highly-directive spatialization with first-order ambisonics](https://mlsanity.com/paper/2602.17732/) (2026)
- [SelectTSL: Prompt-Guided Selective Target Sound Localization in Complex Scenarios](https://mlsanity.com/paper/2607.02343/) (2026)
- [How to Backpropagate through Hungarian in Your DETR?](https://mlsanity.com/paper/2211.14448/) (2022)
- [SHAMaNS: Sound Localization with Hybrid Alpha-Stable Spatial Measure and Neural Steerer](https://mlsanity.com/paper/2506.18954/) (2025)
- [From General-Purpose Audio Tagging to Spatially Grounded Sound Event Localization and Detection](https://mlsanity.com/paper/2606.27751/) (2026)

## Abstract (arXiv)

Data-based and learning-based sound source localization (SSL) has shown promising results in challenging conditions, and is commonly set as a classification or a regression problem. Regression-based approaches have certain advantages over classification-based, such as continuous direction-of-arrival estimation of static and moving sources. However, multi-source scenarios require multiple regressors without a clear training strategy up-to-date, that does not rely on auxiliary information such as simultaneous sound classification. We investigate end-to-end training of such methods with a technique recently proposed for video object detectors, adapted to the SSL setting. A differentiable network is constructed that can be plugged to the output of the localizer to solve the optimal assignment between predictions and references, optimizing directly the popular CLEAR-MOT tracking metrics. Results indicate large improvements over directly optimizing mean squared errors, in terms of localization error, detection metrics, and tracking capabilities.