# wav2pos: Sound Source Localization using Masked Autoencoders

arXiv: https://arxiv.org/abs/2408.15771 · HTML: https://mlsanity.com/paper/2408.15771/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 41 references that are in the map.
- First posted on arXiv 28 Aug 2024.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Masked Autoencoders that Listen](https://mlsanity.com/paper/2207.06405/) (2022)
- [Pyroomacoustics: A Python package for audio room simulations and array processing algorithms](https://mlsanity.com/paper/1710.04196/) (2017)
- [Multimodal Masked Autoencoders Learn Transferable Representations](https://mlsanity.com/paper/2205.14204/) (2022)
- [A Survey of Sound Source Localization with Deep Learning Methods](https://mlsanity.com/paper/2109.03465/) (2021)
- [Robust Sound Source Tracking Using SRP-PHAT and 3D Convolutional Neural Networks](https://mlsanity.com/paper/2006.09006/) (2020)
- [Towards End-to-End Acoustic Localization using Deep Learning: from Audio Signal to Source Position Coordinates](https://mlsanity.com/paper/1807.11094/) (2018)
- [SyncNet: correlating objective for time delay estimation in audio signals](https://mlsanity.com/paper/2203.14639/) (2022)
- [FN-SSL: Full-Band and Narrow-Band Fusion for Sound Source Localization](https://mlsanity.com/paper/2305.19610/) (2023)
- [Extending GCC-PHAT using Shift Equivariant Neural Networks](https://mlsanity.com/paper/2208.04654/) (2022)
- [End-to-end Two-dimensional Sound Source Localization With Ad-hoc Microphone Arrays](https://mlsanity.com/paper/2210.08484/) (2022)
- [Soft Label Coding for End-to-end Sound Source Localization With Ad-hoc Microphone Arrays](https://mlsanity.com/paper/2304.07512/) (2023)
- [Graph neural networks for sound source localization on distributed microphone networks](https://mlsanity.com/paper/2306.16081/) (2023)
- [Dual input neural networks for positional sound source localization](https://mlsanity.com/paper/2308.04169/) (2023)
- [SOLVIT:A Reference-Free Source Localization Technique using Majorization Minimization](https://mlsanity.com/paper/1905.04500/) (2019)
- [The LuViRA Dataset: Synchronized Vision, Radio, and Audio Sensors for Indoor Localization](https://mlsanity.com/paper/2302.05309/) (2023)

## Built on by (most cited first, 1 of 1)

- [Efficient and Microphone-Fault-Tolerant 3D Sound Source Localization](https://mlsanity.com/paper/2505.20961/) (2025)

## Abstract (arXiv)

We present a novel approach to the 3D sound source localization task for distributed ad-hoc microphone arrays by formulating it as a set-to-set regression problem. By training a multi-modal masked autoencoder model that operates on audio recordings and microphone coordinates, we show that such a formulation allows for accurate localization of the sound source, by reconstructing coordinates masked in the input. Our approach is flexible in the sense that a single model can be used with an arbitrary number of microphones, even when a subset of audio recordings and microphone coordinates are missing. We test our method on simulated and real-world recordings of music and speech in indoor environments, and demonstrate competitive performance compared to both classical and other learning based localization methods.