# TF-Mamba: A Time-Frequency Network for Sound Source Localization

arXiv: https://arxiv.org/abs/2409.05034 · HTML: https://mlsanity.com/paper/2409.05034/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Speech and audio SSMs, one of 10 directions in State-space sequence models (S4, Mamba).
- Builds on 27 of its 45 references that are in the map.
- First posted on arXiv 8 Sep 2024.

## Placed in

- State-space sequence models (S4, Mamba) / [Speech and audio SSMs](https://mlsanity.com/ssm/speech-and-audio-ssms/)

## Builds on (most cited first, 27 of 27)

- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model](https://mlsanity.com/paper/2401.09417/) (2024)
- [On the Parameterization and Initialization of Diagonal State Space Models](https://mlsanity.com/paper/2206.11893/) (2022)
- [Jamba: A Hybrid Transformer-Mamba Language Model](https://mlsanity.com/paper/2403.19887/) (2024)
- [Sound Event Localization and Detection of Overlapping Sources Using Convolutional Recurrent Neural Networks](https://mlsanity.com/paper/1807.00129/) (2018)
- [An Investigation of Incorporating Mamba for Speech Enhancement](https://mlsanity.com/paper/2405.06573/) (2024)
- [Mamba in Speech: Towards an Alternative to Self-Attention](https://mlsanity.com/paper/2405.12609/) (2024)
- [gpuRIR: A Python Library for Room Impulse Response Simulation with GPU Acceleration](https://mlsanity.com/paper/1810.11359/) (2018)
- [A Survey of Sound Source Localization with Deep Learning Methods](https://mlsanity.com/paper/2109.03465/) (2021)
- [Dual-path Mamba: Short and Long-term Bidirectional Selective Structured State Space Models for Speech Separation](https://mlsanity.com/paper/2403.18257/) (2024)
- [SSAMBA: Self-Supervised Audio Representation Learning with Mamba State Space Model](https://mlsanity.com/paper/2405.11831/) (2024)
- [Audio Mamba: Bidirectional State Space Model for Audio Representation Learning](https://mlsanity.com/paper/2406.03344/) (2024)
- [Multichannel Long-Term Streaming Neural Speech Enhancement for Static and Moving Speakers](https://mlsanity.com/paper/2403.07675/) (2024)
- [Direction of arrival estimation for multiple sound sources using convolutional recurrent neural network](https://mlsanity.com/paper/1710.10059/) (2017)
- [RawBMamba: End-to-End Bidirectional State Space Model for Audio Deepfake Detection](https://mlsanity.com/paper/2406.06086/) (2024)
- [ADL-MVDR: All deep learning MVDR beamformer for target speech separation](https://mlsanity.com/paper/2008.06994/) (2020)
- [STARSS22: A dataset of spatial recordings of real scenes with spatiotemporal annotations of sound events](https://mlsanity.com/paper/2206.01948/) (2022)
- [Robust Sound Source Tracking Using SRP-PHAT and 3D Convolutional Neural Networks](https://mlsanity.com/paper/2006.09006/) (2020)
- [SALSA-Lite: A Fast and Effective Feature for Polyphonic Sound Event Localization and Detection with Microphone Arrays](https://mlsanity.com/paper/2111.08192/) (2021)
- [FN-SSL: Full-Band and Narrow-Band Fusion for Sound Source Localization](https://mlsanity.com/paper/2305.19610/) (2023)
- [Deep Residual Network for Sound Source Localization in the Time Domain](https://mlsanity.com/paper/1808.06429/) (2018)
- [SRP-DNN: Learning Direct-Path Phase Difference for Multiple Moving Sound Source Localization](https://mlsanity.com/paper/2202.07859/) (2022)
- [Exploiting Deep Neural Networks and Head Movements for Robust Binaural Localisation of Multiple Sources in Reverberant Environments](https://mlsanity.com/paper/1904.03001/) (2019)
- [Dual Knowledge Distillation for Efficient Sound Event Detection](https://mlsanity.com/paper/2402.02781/) (2024)
- [Where's That Voice Coming? Continual Learning for Sound Source Localization](https://mlsanity.com/paper/2407.03661/) (2024)

## Built on by (most cited first, 5 of 5)

- [XLSR-Mamba: A Dual-Column Bidirectional State Space Model for Spoofing Attack Detection](https://mlsanity.com/paper/2411.10027/) (2024)
- [Deep Active Speech Cancellation with Mamba-Masking Network](https://mlsanity.com/paper/2502.01185/) (2025)
- [Autoregressive Guidance of Deep Spatially Selective Filters using Bayesian Tracking for Efficient Extraction of Moving Speakers](https://mlsanity.com/paper/2603.23723/) (2026)
- [Active Speech Enhancement: Active Speech Denoising Decliping and Deveraberation](https://mlsanity.com/paper/2505.16911/) (2025)
- [ESTM: An Enhanced Dual-Branch Spectral-Temporal Mamba for Anomalous Sound Detection](https://mlsanity.com/paper/2509.02471/) (2025)

## Abstract (arXiv)

Sound source localization (SSL) determines the position of sound sources using multi-channel audio data. It is commonly used to improve speech enhancement and separation. Extracting spatial features is crucial for SSL, especially in challenging acoustic environments. Recently, a novel structure referred to as Mamba demonstrated notable performance across various sequence-based modalities. This study introduces the Mamba for SSL tasks. We consider the Mamba-based model to analyze spatial features from speech signals by fusing both time and frequency features, and we develop an SSL system called TF-Mamba. This system integrates time and frequency fusion, with Bidirectional Mamba managing both time-wise and frequency-wise processing. We conduct the experiments on the simulated and real datasets. Experiments show that TF-Mamba significantly outperforms other advanced methods. The code will be publicly released in due course.