# NeuralMUSIC: A Hybrid Neural-Subspace Framework for Robot Sound Source Localization

arXiv: https://arxiv.org/abs/2606.18664 · HTML: https://mlsanity.com/paper/2606.18664/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 30 references that are in the map.
- First posted on arXiv 17 Jun 2026.

## Builds on (most cited first, 14 of 14)

- [Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition](https://mlsanity.com/paper/1804.03209/) (2018)
- [Pyroomacoustics: A Python package for audio room simulations and array processing algorithms](https://mlsanity.com/paper/1710.04196/) (2017)
- [Deep Neural Networks for Multiple Speaker Detection and Localization](https://mlsanity.com/paper/1711.11565/) (2017)
- [DeepMUSIC: Multiple Signal Classification via Deep Learning](https://mlsanity.com/paper/1912.04357/) (2019)
- [DA-MUSIC: Data-Driven DoA Estimation via Deep Augmented MUSIC Algorithm](https://mlsanity.com/paper/2109.10581/) (2021)
- [Self-supervised Neural Audio-Visual Sound Source Localization via Probabilistic Spatial Modeling](https://mlsanity.com/paper/2007.13976/) (2020)
- [SubspaceNet: Deep Learning-Aided Subspace Methods for DoA Estimation](https://mlsanity.com/paper/2306.02271/) (2023)
- [Localization of Simultaneous Moving Sound Sources for Mobile Robot Using a Frequency-Domain Steered Beamformer Approach](https://mlsanity.com/paper/1602.08629/) (2016)
- [AV-PedAware: Self-Supervised Audio-Visual Fusion for Dynamic Pedestrian Awareness](https://mlsanity.com/paper/2411.06789/) (2024)
- [The Un-Kidnappable Robot: Acoustic Localization of Sneaking People](https://mlsanity.com/paper/2310.03743/) (2023)
- [Sound Source Localization for Human-Robot Interaction in Outdoor Environments](https://mlsanity.com/paper/2507.21431/) (2025)
- [FRIDA: FRI-Based DOA Estimation for Arbitrary Array Layouts](https://mlsanity.com/paper/1612.00876/) (2016)
- [SLoClas: A Database for Joint Sound Localization and Classification](https://mlsanity.com/paper/2108.02539/) (2021)
- [BAST: Binaural Audio Spectrogram Transformer for Binaural Sound Localization](https://mlsanity.com/paper/2207.03927/) (2022)

## Abstract (arXiv)

Reliable sound source localization is fundamental to robot audition, enabling autonomous robots to perceive spatial cues and operate effectively in dynamic environments. Classical methods such as Multiple Signal Classification (MUSIC) offer strong theoretical foundations but degrade under low signal-to-noise ratios. While deep learning-based approaches achieve promising performance, they often struggle with limited generalization across conditions. To address these challenges, we propose NeuralMUSIC, a hybrid neural-subspace framework for robotic sound source localization. Specifically, a neural network first estimates the spatial covariance matrix from multichannel microphone observations. The predicted covariance is then integrated into a classical MUSIC pipeline with eigenvalue decomposition (EVD) and pseudo-spectrum computation, followed by a Frequency Attention Fusion (FAF) module to produce the final DOA estimates. To improve data efficiency, we further introduce a Self-supervised Spatial Correlation Learning (SSCL) strategy that leverages unlabeled acoustic data to capture spatial structure. Extensive experiments across different robotic tasks demonstrate that NeuralMUSIC achieves competitive localization accuracy while exhibiting improved robustness and cross-domain generalization.