# Multichannel Long-Term Streaming Neural Speech Enhancement for Static and Moving Speakers

arXiv: https://arxiv.org/abs/2403.07675 · HTML: https://mlsanity.com/paper/2403.07675/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Mar 2024.

## Built on by (most cited first, 28 of 28)

- [Mamba-360: Survey of State Space Models as Transformer Alternative for Long Sequence Modelling: Methods, Applications, and Challenges](https://mlsanity.com/paper/2404.16112/) (2024)
- [Mamba in Speech: Towards an Alternative to Self-Attention](https://mlsanity.com/paper/2405.12609/) (2024)
- [State Space Model for New-Generation Network Alternative to Transformers: A Survey](https://mlsanity.com/paper/2404.09516/) (2024)
- [Dual-path Mamba: Short and Long-term Bidirectional Selective Structured State Space Models for Speech Separation](https://mlsanity.com/paper/2403.18257/) (2024)
- [A Survey of Mamba](https://mlsanity.com/paper/2408.01129/) (2024)
- [SSAMBA: Self-Supervised Audio Representation Learning with Mamba State Space Model](https://mlsanity.com/paper/2405.11831/) (2024)
- [Speech Slytherin: Examining the Performance and Efficiency of Mamba for Speech Separation, Recognition, and Synthesis](https://mlsanity.com/paper/2407.09732/) (2024)
- [TF-Mamba: A Time-Frequency Network for Sound Source Localization](https://mlsanity.com/paper/2409.05034/) (2024)
- [Cross-attention Inspired Selective State Space Models for Target Sound Extraction](https://mlsanity.com/paper/2409.04803/) (2024)
- [Mamba State-Space Models Are Lyapunov-Stable Learners](https://mlsanity.com/paper/2406.00209/) (2024)
- [Schr"odinger Bridge Mamba for One-Step Speech Enhancement](https://mlsanity.com/paper/2510.16834/) (2025)
- [CleanUMamba: A Compact Mamba Network for Speech Denoising using Channel Pruning](https://mlsanity.com/paper/2410.11062/) (2024)
- [Deep Active Speech Cancellation with Mamba-Masking Network](https://mlsanity.com/paper/2502.01185/) (2025)
- [CleanMel: Mel-Spectrogram Enhancement for Improving Both Speech Quality and ASR](https://mlsanity.com/paper/2502.20040/) (2025)
- [Technologies on Effectiveness and Efficiency: A Survey of State Spaces Models](https://mlsanity.com/paper/2503.11224/) (2025)
- [Memba: Membrane-driven Parameter-Efficient Fine-Tuning for Mamba](https://mlsanity.com/paper/2506.18184/) (2025)
- [Damba-ST: Domain-Adaptive Mamba for Efficient Urban Spatio-Temporal Prediction](https://mlsanity.com/paper/2506.18939/) (2025)
- [Adaptive Rotary Steering with Joint Autoregression for Robust Extraction of Closely Moving Speakers in Dynamic Scenarios](https://mlsanity.com/paper/2601.12345/) (2026)
- [Venturing into Uncharted Waters: The Navigation Compass from Transformer to Mamba](https://mlsanity.com/paper/2406.16722/) (2024)
- [Leveraging Joint Spectral and Spatial Learning with MAMBA for Multichannel Speech Enhancement](https://mlsanity.com/paper/2409.10376/) (2024)
- [DeepFilterGAN: A Full-band Real-time Speech Enhancement System with GAN-based Stochastic Regeneration](https://mlsanity.com/paper/2505.23515/) (2025)
- [Autoregressive Guidance of Deep Spatially Selective Filters using Bayesian Tracking for Efficient Extraction of Moving Speakers](https://mlsanity.com/paper/2603.23723/) (2026)
- [VINP: Variational Bayesian Inference with Neural Speech Prior for Joint ASR-Effective Speech Dereverberation and Blind RIR Identification](https://mlsanity.com/paper/2502.07205/) (2025)
- [Active Speech Enhancement: Active Speech Denoising Decliping and Deveraberation](https://mlsanity.com/paper/2505.16911/) (2025)
- [Keyword Mamba: Spoken Keyword Spotting with State Space Models](https://mlsanity.com/paper/2508.07363/) (2025)
- [A Mamba-Based Model for Automatic Chord Recognition](https://mlsanity.com/paper/2601.02101/) (2026)
- [CaSNet: Compress-and-Send Network Based Multi-Device Speech Enhancement Model for Distributed Microphone Arrays](https://mlsanity.com/paper/2601.17711/) (2026)
- [Time-Frequency Weighted Losses for Phoneme Reconstruction in DNN-Based Speech Enhancement](https://mlsanity.com/paper/2606.21635/) (2026)

## Abstract (arXiv)

In this work, we extend our previously proposed offline SpatialNet for long-term streaming multichannel speech enhancement in both static and moving speaker scenarios. SpatialNet exploits spatial information, such as the spatial/steering direction of speech, for discriminating between target speech and interferences, and achieved outstanding performance. The core of SpatialNet is a narrow-band self-attention module used for learning the temporal dynamic of spatial vectors. Towards long-term streaming speech enhancement, we propose to replace the offline self-attention network with online networks that have linear inference complexity w.r.t signal length and meanwhile maintain the capability of learning long-term information. Three variants are developed based on (i) masked self-attention, (ii) Retention, a self-attention variant with linear inference complexity, and (iii) Mamba, a structured-state-space-based RNN-like network. Moreover, we investigate the length extrapolation ability of different networks, namely test on signals that are much longer than training signals, and propose a short-signal training plus long-signal fine-tuning strategy, which largely improves the length extrapolation ability of the networks within limited training time. Overall, the proposed online SpatialNet achieves outstanding speech enhancement performance for long audio streams, and for both static and moving speakers. The proposed method is open-sourced in https://github.com/Audio-WestlakeU/NBSS.