# Multi-microphone Complex Spectral Mapping for Utterance-wise and Continuous Speech Separation

arXiv: https://arxiv.org/abs/2010.01703 · HTML: https://mlsanity.com/paper/2010.01703/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Oct 2020.

## Built on by (most cited first, 9 of 9)

- [VarArray: Array-Geometry-Agnostic Continuous Speech Separation](https://mlsanity.com/paper/2110.05745/) (2021)
- [Implicit Neural Spatial Filtering for Multichannel Source Separation in the Waveform Domain](https://mlsanity.com/paper/2206.15423/) (2022)
- [Continuous Speech Separation with Ad Hoc Microphone Arrays](https://mlsanity.com/paper/2103.02378/) (2021)
- [UNSSOR: Unsupervised Neural Speech Separation by Leveraging Over-determined Training Mixtures](https://mlsanity.com/paper/2305.20054/) (2023)
- [Mixture Encoder for Joint Speech Separation and Recognition](https://mlsanity.com/paper/2306.12173/) (2023)
- [A Survey of Deep Learning for Complex Speech Spectrograms](https://mlsanity.com/paper/2505.08694/) (2025)
- [A Time-domain Real-valued Generalized Wiener Filter for Multi-channel Neural Separation Systems](https://mlsanity.com/paper/2112.03533/) (2021)
- [On monoaural speech enhancement for automatic recognition of real noisy speech using mixture invariant training](https://mlsanity.com/paper/2205.01751/) (2022)
- [CabinSep: IR-Augmented Mask-Based MVDR for Real-Time In-Car Speech Separation with Distributed Heterogeneous Arrays](https://mlsanity.com/paper/2509.01399/) (2025)

## Abstract (arXiv)

We propose multi-microphone complex spectral mapping, a simple way of applying deep learning for time-varying non-linear beamforming, for speaker separation in reverberant conditions. We aim at both speaker separation and dereverberation. Our study first investigates offline utterance-wise speaker separation and then extends to block-online continuous speech separation (CSS). Assuming a fixed array geometry between training and testing, we train deep neural networks (DNN) to predict the real and imaginary (RI) components of target speech at a reference microphone from the RI components of multiple microphones. We then integrate multi-microphone complex spectral mapping with minimum variance distortionless response (MVDR) beamforming and post-filtering to further improve separation, and combine it with frame-level speaker counting for block-online CSS. Although our system is trained on simulated room impulse responses (RIR) based on a fixed number of microphones arranged in a given geometry, it generalizes well to a real array with the same geometry. State-of-the-art separation performance is obtained on the simulated two-talker SMS-WSJ corpus and the real-recorded LibriCSS dataset.