# VarArray: Array-Geometry-Agnostic Continuous Speech Separation

arXiv: https://arxiv.org/abs/2110.05745 · HTML: https://mlsanity.com/paper/2110.05745/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 33 references that are in the map.
- First posted on arXiv 12 Oct 2021.

## Builds on (most cited first, 22 of 22)

- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [MUSAN: A Music, Speech, and Noise Corpus](https://mlsanity.com/paper/1510.08484/) (2015)
- [Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition](https://mlsanity.com/paper/2010.10504/) (2020)
- [Dual-path RNN: efficient long sequence modeling for time-domain single-channel speech separation](https://mlsanity.com/paper/1910.06379/) (2019)
- [Multi-talker Speech Separation with Utterance-level Permutation Invariant Training of Deep Recurrent Neural Networks](https://mlsanity.com/paper/1703.06284/) (2017)
- [Continuous speech separation: dataset and analysis](https://mlsanity.com/paper/2001.11482/) (2020)
- [Earnings-21: A Practical Benchmark for ASR in the Wild](https://mlsanity.com/paper/2104.11348/) (2021)
- [MIMO-SPEECH: End-to-End Multi-Channel Multi-Speaker Speech Recognition](https://mlsanity.com/paper/1910.06522/) (2019)
- [End-to-End Multi-speaker Speech Recognition with Transformer](https://mlsanity.com/paper/2002.03921/) (2020)
- [Continuous Speech Separation with Conformer](https://mlsanity.com/paper/2008.05773/) (2020)
- [End-to-end Microphone Permutation and Number Invariant Multi-channel Speech Separation](https://mlsanity.com/paper/1910.14104/) (2019)
- [On the limit of English conversational speech recognition](https://mlsanity.com/paper/2105.00982/) (2021)
- [Recognizing Overlapped Speech in Meetings: A Multichannel Separation Approach Using Neural Networks](https://mlsanity.com/paper/1810.03655/) (2018)
- [Advances in Online Audio-Visual Meeting Transcription](https://mlsanity.com/paper/1912.04979/) (2019)
- [WER we are and WER we think we are](https://mlsanity.com/paper/2010.03432/) (2020)
- [End-to-End Dereverberation, Beamforming, and Speech Recognition with Improved Numerical Stability and Advanced Frontend](https://mlsanity.com/paper/2102.11525/) (2021)
- [Multi-microphone Complex Spectral Mapping for Utterance-wise and Continuous Speech Separation](https://mlsanity.com/paper/2010.01703/) (2020)
- [Exploring End-to-End Multi-channel ASR with Bias Information for Meeting Transcription](https://mlsanity.com/paper/2011.03110/) (2020)
- [Graph-PIT: Generalized permutation invariant training for continuous separation of arbitrary numbers of speakers](https://mlsanity.com/paper/2107.14446/) (2021)
- [Low-Latency Speaker-Independent Continuous Speech Separation](https://mlsanity.com/paper/1904.06478/) (2019)
- [Neural Speech Separation Using Spatially Distributed Microphones](https://mlsanity.com/paper/2004.13670/) (2020)
- [Microphone Array Generalization for Multichannel Narrowband Deep Speech Enhancement](https://mlsanity.com/paper/2107.12601/) (2021)

## Built on by (most cited first, 10 of 10)

- [The CHiME-7 DASR Challenge: Distant Meeting Transcription with Multiple Devices in Diverse Scenarios](https://mlsanity.com/paper/2306.13734/) (2023)
- [One model to rule them all ? Towards End-to-End Joint Speaker Diarization and Speech Recognition](https://mlsanity.com/paper/2310.01688/) (2023)
- [VarArray Meets t-SOT: Advancing the State of the Art of Streaming Distant Conversational Speech Recognition](https://mlsanity.com/paper/2209.04974/) (2022)
- [ArrayDPS: Unsupervised Blind Speech Separation with a Diffusion Prior](https://mlsanity.com/paper/2505.05657/) (2025)
- [Leveraging Real Conversational Data for Multi-Channel Continuous Speech Separation](https://mlsanity.com/paper/2204.03232/) (2022)
- [Handling Trade-Offs in Speech Separation with Sparsely-Gated Mixture of Experts](https://mlsanity.com/paper/2211.06493/) (2022)
- [Binaural Angular Separation Network](https://mlsanity.com/paper/2401.08864/) (2024)
- [PickNet: Real-Time Channel Selection for Ad Hoc Microphone Arrays](https://mlsanity.com/paper/2201.09586/) (2022)
- [Mutual Learning of Single- and Multi-Channel End-to-End Neural Diarization](https://mlsanity.com/paper/2210.03459/) (2022)
- [SpatialEmb: Extract and Encode Spatial Information for 1-Stage Multi-channel Multi-speaker ASR on Arbitrary Microphone Arrays](https://mlsanity.com/paper/2601.18037/) (2026)

## Abstract (arXiv)

Continuous speech separation using a microphone array was shown to be promising in dealing with the speech overlap problem in natural conversation transcription. This paper proposes VarArray, an array-geometry-agnostic speech separation neural network model. The proposed model is applicable to any number of microphones without retraining while leveraging the nonlinear correlation between the input channels. The proposed method adapts different elements that were proposed before separately, including transform-average-concatenate, conformer speech separation, and inter-channel phase differences, and combines them in an efficient and cohesive way. Large-scale evaluation was performed with two real meeting transcription tasks by using a fully developed transcription system requiring no prior knowledge such as reference segmentations, which allowed us to measure the impact that the continuous speech separation system could have in realistic settings. The proposed model outperformed a previous approach to array-geometry-agnostic modeling for all of the geometry configurations considered, achieving asclite-based speaker-agnostic word error rates of 17.5% and 20.4% for the AMI development and evaluation sets, respectively, in the end-to-end setting using no ground-truth segmentations.