# Multi-channel Speech Separation Using Spatially Selective Deep Non-linear Filters

arXiv: https://arxiv.org/abs/2304.12023 · HTML: https://mlsanity.com/paper/2304.12023/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 54 references that are in the map.
- First posted on arXiv 24 Apr 2023.

## Builds on (most cited first, 20 of 20)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Show and Tell: A Neural Image Caption Generator](https://mlsanity.com/paper/1411.4555/) (2014)
- [MUSAN: A Music, Speech, and Noise Corpus](https://mlsanity.com/paper/1510.08484/) (2015)
- [SDR - half-baked or well done?](https://mlsanity.com/paper/1811.02508/) (2018)
- [DNSMOS P.835: A Non-Intrusive Perceptual Objective Speech Quality Metric to Evaluate Noise Suppressors](https://mlsanity.com/paper/2110.01763/) (2021)
- [Pyroomacoustics: A Python package for audio room simulations and array processing algorithms](https://mlsanity.com/paper/1710.04196/) (2017)
- [Multi-talker Speech Separation with Utterance-level Permutation Invariant Training of Deep Recurrent Neural Networks](https://mlsanity.com/paper/1703.06284/) (2017)
- [End-to-end Microphone Permutation and Number Invariant Multi-channel Speech Separation](https://mlsanity.com/paper/1910.14104/) (2019)
- [The Cone of Silence: Speech Separation by Localization](https://mlsanity.com/paper/2010.06007/) (2020)
- [Channel-Attention Dense U-Net for Multichannel Speech Enhancement](https://mlsanity.com/paper/2001.11542/) (2020)
- [Insights Into Deep Non-linear Filters for Improved Multi-channel Speech Enhancement](https://mlsanity.com/paper/2206.13310/) (2022)
- [Enhancing End-to-End Multi-channel Speech Separation via Spatial Feature Learning](https://mlsanity.com/paper/2003.03927/) (2020)
- [McNet: Fuse Multiple Cues for Multichannel Speech Enhancement](https://mlsanity.com/paper/2211.08872/) (2022)
- [Nonlinear Spatial Filtering in Multichannel Speech Enhancement](https://mlsanity.com/paper/2104.11033/) (2021)
- [On the Role of Spatial, Spectral, and Temporal Processing for DNN-based Non-linear Multi-channel Speech Enhancement](https://mlsanity.com/paper/2206.11181/) (2022)
- [Implicit Neural Spatial Filtering for Multichannel Source Separation in the Waveform Domain](https://mlsanity.com/paper/2206.15423/) (2022)
- [Spatially Selective Deep Non-linear Filters for Speaker Extraction](https://mlsanity.com/paper/2211.02420/) (2022)
- [Multi-Microphone Speaker Separation by Spatial Regions](https://mlsanity.com/paper/2303.07143/) (2023)
- [Multichannel Speech Separation with Narrow-band Conformer](https://mlsanity.com/paper/2204.04464/) (2022)
- [Analyzing the impact of speaker localization errors on speech separation for automatic speech recognition](https://mlsanity.com/paper/1910.11114/) (2019)

## Built on by (most cited first, 10 of 10)

- [Score Distillation Sampling for Audio: Source Separation, Synthesis, and Beyond](https://mlsanity.com/paper/2505.04621/) (2025)
- [xLSTM-SENet: xLSTM for Single-Channel Speech Enhancement](https://mlsanity.com/paper/2501.06146/) (2025)
- [Steering Deep Non-Linear Spatially Selective Filters for Weakly Guided Extraction of Moving Speakers in Dynamic Scenarios](https://mlsanity.com/paper/2505.14517/) (2025)
- [Multichannel-to-Multichannel Target Sound Extraction Using Direction and Timestamp Clues](https://mlsanity.com/paper/2409.12415/) (2024)
- [Self-Steering Deep Non-Linear Spatially Selective Filters for Efficient Extraction of Moving Speakers under Weak Guidance](https://mlsanity.com/paper/2507.02791/) (2025)
- [Adaptive Rotary Steering with Joint Autoregression for Robust Extraction of Closely Moving Speakers in Dynamic Scenarios](https://mlsanity.com/paper/2601.12345/) (2026)
- [Autoregressive Guidance of Deep Spatially Selective Filters using Bayesian Tracking for Efficient Extraction of Moving Speakers](https://mlsanity.com/paper/2603.23723/) (2026)
- [Mask-Weighted Spatial Likelihood Coding for Speaker-Independent Joint Localization and Mask Estimation](https://mlsanity.com/paper/2410.19595/) (2024)
- [SoundCompass: Navigating Target Sound Extraction With Effective Directional Clue Integration In Complex Acoustic Scenes](https://mlsanity.com/paper/2509.18561/) (2025)
- [Weakly Guided and Autoregressive Beamformer Parameterization for Generalizable Moving Speaker Extraction in Higher-Order Ambisonics](https://mlsanity.com/paper/2607.04471/) (2026)

## Abstract (arXiv)

In a multi-channel separation task with multiple speakers, we aim to recover all individual speech signals from the mixture. In contrast to single-channel approaches, which rely on the different spectro-temporal characteristics of the speech signals, multi-channel approaches should additionally utilize the different spatial locations of the sources for a more powerful separation especially when the number of sources increases. To enhance the spatial processing in a multi-channel source separation scenario, in this work, we propose a deep neural network (DNN) based spatially selective filter (SSF) that can be spatially steered to extract the speaker of interest by initializing a recurrent neural network layer with the target direction. We compare the proposed SSF with a common end-to-end direct separation (DS) approach trained using utterance-wise permutation invariant training (PIT), which only implicitly learns to perform spatial filtering. We show that the SSF has a clear advantage over a DS approach with the same underlying network architecture when there are more than two speakers in the mixture, which can be attributed to a better use of the spatial information. Furthermore, we find that the SSF generalizes much better to additional noise sources that were not seen during training and to scenarios with speakers positioned at a similar angle.