# Recognizing Overlapped Speech in Meetings: A Multichannel Separation Approach Using Neural Networks

arXiv: https://arxiv.org/abs/1810.03655 · HTML: https://mlsanity.com/paper/1810.03655/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Oct 2018.

## Built on by (most cited first, 23 of 23)

- [A Review of Speaker Diarization: Recent Advances with Deep Learning](https://mlsanity.com/paper/2101.09624/) (2021)
- [Streaming Multi-Talker ASR with Token-Level Serialized Output Training](https://mlsanity.com/paper/2202.00842/) (2022)
- [Streaming Multi-talker Speech Recognition with Joint Speaker Identification](https://mlsanity.com/paper/2104.02109/) (2021)
- [VarArray: Array-Geometry-Agnostic Continuous Speech Separation](https://mlsanity.com/paper/2110.05745/) (2021)
- [VarArray Meets t-SOT: Advancing the State of the Art of Streaming Distant Conversational Speech Recognition](https://mlsanity.com/paper/2209.04974/) (2022)
- [Typing to Listen at the Cocktail Party: Text-Guided Target Speaker Extraction](https://mlsanity.com/paper/2310.07284/) (2023)
- [Continuous Speech Separation with Ad Hoc Microphone Arrays](https://mlsanity.com/paper/2103.02378/) (2021)
- [Directed Speech Separation for Automatic Speech Recognition of Long Form Conversational Speech](https://mlsanity.com/paper/2112.05863/) (2021)
- [Speech separation with large-scale self-supervised learning](https://mlsanity.com/paper/2211.05172/) (2022)
- [Leveraging Real Conversational Data for Multi-Channel Continuous Speech Separation](https://mlsanity.com/paper/2204.03232/) (2022)
- [Audio-visual End-to-end Multi-channel Speech Separation, Dereverberation and Recognition](https://mlsanity.com/paper/2307.02909/) (2023)
- [Audio-visual multi-channel speech separation, dereverberation and recognition](https://mlsanity.com/paper/2204.01977/) (2022)
- [Handling Trade-Offs in Speech Separation with Sparsely-Gated Mixture of Experts](https://mlsanity.com/paper/2211.06493/) (2022)
- [MedleyVox: An Evaluation Dataset for Multiple Singing Voices Separation](https://mlsanity.com/paper/2211.07302/) (2022)
- [Thinking in Directivity: Speech Large Language Model for Multi-Talker Directional Speech Recognition](https://mlsanity.com/paper/2506.14973/) (2025)
- [BeamTransformer: Microphone Array-based Overlapping Speech Detection](https://mlsanity.com/paper/2109.04049/) (2021)
- [Combining TF-GridNet and Mixture Encoder for Continuous Speech Separation for Meeting Transcription](https://mlsanity.com/paper/2309.08454/) (2023)
- [Time-Frequency-Based Attention Cache Memory Model for Real-Time Speech Separation](https://mlsanity.com/paper/2505.13094/) (2025)
- [Error Analysis in a Modular Meeting Transcription System](https://mlsanity.com/paper/2509.10143/) (2025)
- [Blind Speech Separation and Dereverberation using Neural Beamforming](https://mlsanity.com/paper/2103.13443/) (2021)
- [Joint Beamforming and Speaker-Attributed ASR for Real Distant-Microphone Meeting Transcription](https://mlsanity.com/paper/2410.21849/) (2024)
- [CabinSep: IR-Augmented Mask-Based MVDR for Real-Time In-Car Speech Separation with Distributed Heterogeneous Arrays](https://mlsanity.com/paper/2509.01399/) (2025)
- [G-STAR: End-to-End Global Speaker-Tracking Attributed Recognition](https://mlsanity.com/paper/2603.10468/) (2026)

## Abstract (arXiv)

The goal of this work is to develop a meeting transcription system that can recognize speech even when utterances of different speakers are overlapped. While speech overlaps have been regarded as a major obstacle in accurately transcribing meetings, a traditional beamformer with a single output has been exclusively used because previously proposed speech separation techniques have critical constraints for application to real meetings. This paper proposes a new signal processing module, called an unmixing transducer, and describes its implementation using a windowed BLSTM. The unmixing transducer has a fixed number, say J, of output channels, where J may be different from the number of meeting attendees, and transforms an input multi-channel acoustic signal into J time-synchronous audio streams. Each utterance in the meeting is separated and emitted from one of the output channels. Then, each output signal can be simply fed to a speech recognition back-end for segmentation and transcription. Our meeting transcription system using the unmixing transducer outperforms a system based on a state-of-the-art neural mask-based beamformer by 10.8%. Significant improvements are observed in overlapped segments. To the best of our knowledge, this is the first report that applies overlapped speech recognition to unconstrained real meeting audio.