# Attention-Driven Multichannel Speech Enhancement in Moving Sound Source Scenarios

arXiv: https://arxiv.org/abs/2312.10756 · HTML: https://mlsanity.com/paper/2312.10756/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 31 references that are in the map.
- First posted on arXiv 17 Dec 2023.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separation](https://mlsanity.com/paper/1809.07454/) (2018)
- [Deep Learning for Audio Signal Processing](https://mlsanity.com/paper/1905.00078/) (2019)
- [Sound Event Localization and Detection of Overlapping Sources Using Convolutional Recurrent Neural Networks](https://mlsanity.com/paper/1807.00129/) (2018)
- [gpuRIR: A Python Library for Room Impulse Response Simulation with GPU Acceleration](https://mlsanity.com/paper/1810.11359/) (2018)
- [FaSNet: Low-latency Adaptive Beamforming for Multi-microphone Audio Processing](https://mlsanity.com/paper/1909.13387/) (2019)
- [Channel-Attention Dense U-Net for Multichannel Speech Enhancement](https://mlsanity.com/paper/2001.11542/) (2020)
- [Implicit Neural Spatial Filtering for Multichannel Source Separation in the Waveform Domain](https://mlsanity.com/paper/2206.15423/) (2022)
- [Mask-based Neural Beamforming for Moving Speakers with Self-Attention-based Tracking](https://mlsanity.com/paper/2205.03568/) (2022)
- [NICE-Beam: Neural Integrated Covariance Estimators for Time-Varying Beamformers](https://mlsanity.com/paper/2112.04613/) (2021)
- [Separation of Moving Sound Sources Using Multichannel NMF and Acoustic Tracking](https://mlsanity.com/paper/1710.10005/) (2017)
- [Exploring Optimal DNN Architecture for End-to-End Beamformers Based on Time-frequency References](https://mlsanity.com/paper/2005.12683/) (2020)
- [Mobile Microphone Array Speech Detection and Localization in Diverse Everyday Environments](https://mlsanity.com/paper/2106.14787/) (2021)

## Abstract (arXiv)

Current multichannel speech enhancement algorithms typically assume a stationary sound source, a common mismatch with reality that limits their performance in real-world scenarios. This paper focuses on attention-driven spatial filtering techniques designed for dynamic settings. Specifically, we study the application of linear and nonlinear attention-based methods for estimating time-varying spatial covariance matrices used to design the filters. We also investigate the direct estimation of spatial filters by attention-based methods without explicitly estimating spatial statistics. The clean speech clips from WSJ0 are employed for simulating speech signals of moving speakers in a reverberant environment. The experimental dataset is built by mixing the simulated speech signals with multichannel real noise from CHiME-3. Evaluation results show that the attention-driven approaches are robust and consistently outperform conventional spatial filtering approaches in both static and dynamic sound environments.