# End-to-end Microphone Permutation and Number Invariant Multi-channel Speech Separation

arXiv: https://arxiv.org/abs/1910.14104 · HTML: https://mlsanity.com/paper/1910.14104/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Oct 2019.

## Built on by (most cited first, 30 of 31)

- [A Survey of Sound Source Localization with Deep Learning Methods](https://mlsanity.com/paper/2109.03465/) (2021)
- [Insights Into Deep Non-linear Filters for Improved Multi-channel Speech Enhancement](https://mlsanity.com/paper/2206.13310/) (2022)
- [VarArray: Array-Geometry-Agnostic Continuous Speech Separation](https://mlsanity.com/paper/2110.05745/) (2021)
- [Multi-channel Speech Separation Using Spatially Selective Deep Non-linear Filters](https://mlsanity.com/paper/2304.12023/) (2023)
- [ReZero: Region-customizable Sound Extraction](https://mlsanity.com/paper/2308.16892/) (2023)
- [Teacher-Student MixIT for Unsupervised and Semi-supervised Speech Separation](https://mlsanity.com/paper/2106.07843/) (2021)
- [Subnetwork-to-go: Elastic Neural Network with Dynamic Training and Customizable Inference](https://mlsanity.com/paper/2312.03464/) (2023)
- [Hybrid Neural Networks for On-device Directional Hearing](https://mlsanity.com/paper/2112.05893/) (2021)
- [VarArray Meets t-SOT: Advancing the State of the Art of Streaming Distant Conversational Speech Recognition](https://mlsanity.com/paper/2209.04974/) (2022)
- [ArrayDPS: Unsupervised Blind Speech Separation with a Diffusion Prior](https://mlsanity.com/paper/2505.05657/) (2025)
- [Implicit Neural Spatial Filtering for Multichannel Source Separation in the Waveform Domain](https://mlsanity.com/paper/2206.15423/) (2022)
- [Neural Blind Source Separation and Diarization for Distant Speech Recognition](https://mlsanity.com/paper/2406.08396/) (2024)
- [Continuous Speech Separation with Ad Hoc Microphone Arrays](https://mlsanity.com/paper/2103.02378/) (2021)
- [Gull: A Generative Multifunctional Audio Codec](https://mlsanity.com/paper/2404.04947/) (2024)
- [Learning to Rank Microphones for Distant Speech Recognition](https://mlsanity.com/paper/2104.02819/) (2021)
- [Multi-Channel End-to-End Neural Diarization with Distributed Microphones](https://mlsanity.com/paper/2110.04694/) (2021)
- [A Two-Stage Band-Split Mamba-2 Network For Music Separation](https://mlsanity.com/paper/2409.06245/) (2024)
- [Learning Filterbanks for End-to-End Acoustic Beamforming](https://mlsanity.com/paper/2111.04614/) (2021)
- [A Time-domain Real-valued Generalized Wiener Filter for Multi-channel Neural Separation Systems](https://mlsanity.com/paper/2112.03533/) (2021)
- [Guided Speech Enhancement Network](https://mlsanity.com/paper/2303.07486/) (2023)
- [Enhancing Mobile Privacy and Security: A Face Skin Patch-Based Anti-Spoofing Approach](https://mlsanity.com/paper/2308.04798/) (2023)
- [PickNet: Real-Time Channel Selection for Ad Hoc Microphone Arrays](https://mlsanity.com/paper/2201.09586/) (2022)
- [Mutual Learning of Single- and Multi-Channel End-to-End Neural Diarization](https://mlsanity.com/paper/2210.03459/) (2022)
- [MIMO-DBnet: Multi-channel Input and Multiple Outputs DOA-aware Beamforming Network for Speech Separation](https://mlsanity.com/paper/2212.03401/) (2022)
- [Addressing Feature Imbalance in Sound Source Separation](https://mlsanity.com/paper/2309.05287/) (2023)
- [Joint Beamforming and Speaker-Attributed ASR for Real Distant-Microphone Meeting Transcription](https://mlsanity.com/paper/2410.21849/) (2024)
- [CabinSep: IR-Augmented Mask-Based MVDR for Real-Time In-Car Speech Separation with Distributed Heterogeneous Arrays](https://mlsanity.com/paper/2509.01399/) (2025)
- [Multi-channel multi-speaker transformer for speech recognition](https://mlsanity.com/paper/2601.02688/) (2026)
- [CaSNet: Compress-and-Send Network Based Multi-Device Speech Enhancement Model for Distributed Microphone Arrays](https://mlsanity.com/paper/2601.17711/) (2026)
- [SpatialEmb: Extract and Encode Spatial Information for 1-Stage Multi-channel Multi-speaker ASR on Arbitrary Microphone Arrays](https://mlsanity.com/paper/2601.18037/) (2026)

## Abstract (arXiv)

An important problem in ad-hoc microphone speech separation is how to guarantee the robustness of a system with respect to the locations and numbers of microphones. The former requires the system to be invariant to different indexing of the microphones with the same locations, while the latter requires the system to be able to process inputs with varying dimensions. Conventional optimization-based beamforming techniques satisfy these requirements by definition, while for deep learning-based end-to-end systems those constraints are not fully addressed. In this paper, we propose transform-average-concatenate (TAC), a simple design paradigm for channel permutation and number invariant multi-channel speech separation. Based on the filter-and-sum network (FaSNet), a recently proposed end-to-end time-domain beamforming system, we show how TAC significantly improves the separation performance across various numbers of microphones in noisy reverberant separation tasks with ad-hoc arrays. Moreover, we show that TAC also significantly improves the separation performance with fixed geometry array configuration, further proving the effectiveness of the proposed paradigm in the general problem of multi-microphone speech separation.