# Continuous Speech Separation with Ad Hoc Microphone Arrays

arXiv: https://arxiv.org/abs/2103.02378 · HTML: https://mlsanity.com/paper/2103.02378/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 29 references that are in the map.
- First posted on arXiv 3 Mar 2021.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Permutation Invariant Training of Deep Models for Speaker-Independent Multi-talker Speech Separation](https://mlsanity.com/paper/1607.00325/) (2016)
- [Continuous speech separation: dataset and analysis](https://mlsanity.com/paper/2001.11482/) (2020)
- [gpuRIR: A Python Library for Room Impulse Response Simulation with GPU Acceleration](https://mlsanity.com/paper/1810.11359/) (2018)
- [MIMO-SPEECH: End-to-End Multi-Channel Multi-Speaker Speech Recognition](https://mlsanity.com/paper/1910.06522/) (2019)
- [Continuous Speech Separation with Conformer](https://mlsanity.com/paper/2008.05773/) (2020)
- [End-to-end Microphone Permutation and Number Invariant Multi-channel Speech Separation](https://mlsanity.com/paper/1910.14104/) (2019)
- [Recognizing Overlapped Speech in Meetings: A Multichannel Separation Approach Using Neural Networks](https://mlsanity.com/paper/1810.03655/) (2018)
- [Advances in Online Audio-Visual Meeting Transcription](https://mlsanity.com/paper/1912.04979/) (2019)
- [Interactive Speech and Noise Modeling for Speech Enhancement](https://mlsanity.com/paper/2012.09408/) (2020)
- [Multi-microphone Complex Spectral Mapping for Utterance-wise and Continuous Speech Separation](https://mlsanity.com/paper/2010.01703/) (2020)
- [A comprehensive study of speech separation: spectrogram vs waveform separation](https://mlsanity.com/paper/1905.07497/) (2019)
- [Neural Speech Separation Using Spatially Distributed Microphones](https://mlsanity.com/paper/2004.13670/) (2020)
- [Utterance-Wise Meeting Transcription System Using Asynchronous Distributed Microphones](https://mlsanity.com/paper/2007.15868/) (2020)
- [Distributed speech separation in spatially unconstrained microphone arrays](https://mlsanity.com/paper/2011.00982/) (2020)
- [Block-Online Guided Source Separation](https://mlsanity.com/paper/2011.07791/) (2020)
- [Deep Ad-hoc Beamforming Based on Speaker Extraction for Target-Dependent Speech Separation](https://mlsanity.com/paper/2012.00403/) (2020)

## Built on by (most cited first, 5 of 5)

- [MIMO Self-attentive RNN Beamformer for Multi-speaker Speech Separation](https://mlsanity.com/paper/2104.08450/) (2021)
- [Multi-Channel End-to-End Neural Diarization with Distributed Microphones](https://mlsanity.com/paper/2110.04694/) (2021)
- [PickNet: Real-Time Channel Selection for Ad Hoc Microphone Arrays](https://mlsanity.com/paper/2201.09586/) (2022)
- [LibriWASN: A Data Set for Meeting Separation, Diarization, and Recognition with Asynchronous Recording Devices](https://mlsanity.com/paper/2308.10682/) (2023)
- [Multi-channel multi-speaker transformer for speech recognition](https://mlsanity.com/paper/2601.02688/) (2026)

## Abstract (arXiv)

Speech separation has been shown effective for multi-talker speech recognition. Under the ad hoc microphone array setup where the array consists of spatially distributed asynchronous microphones, additional challenges must be overcome as the geometry and number of microphones are unknown beforehand. Prior studies show, with a spatial-temporalinterleaving structure, neural networks can efficiently utilize the multi-channel signals of the ad hoc array. In this paper, we further extend this approach to continuous speech separation. Several techniques are introduced to enable speech separation for real continuous recordings. First, we apply a transformer-based network for spatio-temporal modeling of the ad hoc array signals. In addition, two methods are proposed to mitigate a speech duplication problem during single talker segments, which seems more severe in the ad hoc array scenarios. One method is device distortion simulation for reducing the acoustic mismatch between simulated training data and real recordings. The other is speaker counting to detect the single speaker segments and merge the output signal channels. Experimental results for AdHoc-LibiCSS, a new dataset consisting of continuous recordings of concatenated LibriSpeech utterances obtained by multiple different devices, show the proposed separation method can significantly improve the ASR accuracy for overlapped speech with little performance degradation for single talker segments.