# Integration of speech separation, diarization, and recognition for multi-speaker meetings: System description, comparison, and analysis

arXiv: https://arxiv.org/abs/2011.02014 · HTML: https://mlsanity.com/paper/2011.02014/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Nov 2020.

## Built on by (most cited first, 30 of 35)

- [A Review of Speaker Diarization: Recent Advances with Deep Learning](https://mlsanity.com/paper/2101.09624/) (2021)
- [Streaming Multi-Talker ASR with Token-Level Serialized Output Training](https://mlsanity.com/paper/2202.00842/) (2022)
- [End-to-End Speaker-Attributed ASR with Transformer](https://mlsanity.com/paper/2104.02128/) (2021)
- [Large-Scale Pre-Training of End-to-End Multi-Talker ASR for Meeting Transcription with Single Distant Microphone](https://mlsanity.com/paper/2103.16776/) (2021)
- [Transcribe-to-Diarize: Neural Speaker Diarization for Unlimited Number of Speakers using End-to-End Speaker-Attributed ASR](https://mlsanity.com/paper/2110.03151/) (2021)
- [The CHiME-7 DASR Challenge: Distant Meeting Transcription with Multiple Devices in Diverse Scenarios](https://mlsanity.com/paper/2306.13734/) (2023)
- [NOTSOFAR-1 Challenge: New Datasets, Baseline, and Tasks for Distant Meeting Transcription](https://mlsanity.com/paper/2401.08887/) (2024)
- [SpeakerLM: End-to-End Versatile Speaker Diarization and Recognition with Multimodal Large Language Models](https://mlsanity.com/paper/2508.06372/) (2025)
- [Streaming Speaker-Attributed ASR with Token-Level Speaker Embeddings](https://mlsanity.com/paper/2203.16685/) (2022)
- [One model to rule them all ? Towards End-to-End Joint Speaker Diarization and Speech Recognition](https://mlsanity.com/paper/2310.01688/) (2023)
- [A Comparative Study of Modular and Joint Approaches for Speaker-Attributed ASR on Monaural Long-Form Audio](https://mlsanity.com/paper/2107.02852/) (2021)
- [Simulating realistic speech overlaps improves multi-talker ASR](https://mlsanity.com/paper/2210.15715/) (2022)
- [VarArray Meets t-SOT: Advancing the State of the Art of Streaming Distant Conversational Speech Recognition](https://mlsanity.com/paper/2209.04974/) (2022)
- [Typing to Listen at the Cocktail Party: Text-Guided Target Speaker Extraction](https://mlsanity.com/paper/2310.07284/) (2023)
- [D-ORCA: Dialogue-Centric Optimization for Robust Audio-Visual Captioning](https://mlsanity.com/paper/2602.07960/) (2026)
- [Unifying Diarization, Separation, and ASR with Multi-Speaker Encoder](https://mlsanity.com/paper/2508.20474/) (2025)
- [Leveraging Real Conversational Data for Multi-Channel Continuous Speech Separation](https://mlsanity.com/paper/2204.03232/) (2022)
- [MSA-ASR: Efficient Multilingual Speaker Attribution with frozen ASR Models](https://mlsanity.com/paper/2411.18152/) (2024)
- [Disentangling the Impacts of Language and Channel Variability on Speech Separation Networks](https://mlsanity.com/paper/2203.16040/) (2022)
- [Handling Trade-Offs in Speech Separation with Sparsely-Gated Mixture of Experts](https://mlsanity.com/paper/2211.06493/) (2022)
- [CasNet: Investigating Channel Robustness for Speech Separation](https://mlsanity.com/paper/2210.15370/) (2022)
- [Reverberation as Supervision for Speech Separation](https://mlsanity.com/paper/2211.08303/) (2022)
- [End-to-End Integration of Speech Separation and Voice Activity Detection for Low-Latency Diarization of Telephone Conversations](https://mlsanity.com/paper/2303.12002/) (2023)
- [On Data Sampling Strategies for Training Neural Network Speech Separation Models](https://mlsanity.com/paper/2304.07142/) (2023)
- [Recent Trends in Distant Conversational Speech Recognition: A Review of CHiME-7 and 8 DASR Challenges](https://mlsanity.com/paper/2507.18161/) (2025)
- [Analysis of impact of emotions on target speech extraction and speech separation](https://mlsanity.com/paper/2208.07091/) (2022)
- [A Teacher-Student approach for extracting informative speaker embeddings from speech mixtures](https://mlsanity.com/paper/2306.00634/) (2023)
- [Combining TF-GridNet and Mixture Encoder for Continuous Speech Separation for Meeting Transcription](https://mlsanity.com/paper/2309.08454/) (2023)
- [Error Analysis in a Modular Meeting Transcription System](https://mlsanity.com/paper/2509.10143/) (2025)
- [MK-SGC-SC: Multiple Kernel Guided Sparse Graph Construction in Spectral Clustering for Unsupervised Speaker Diarization](https://mlsanity.com/paper/2601.19946/) (2026)

## Abstract (arXiv)

Multi-speaker speech recognition of unsegmented recordings has diverse applications such as meeting transcription and automatic subtitle generation. With technical advances in systems dealing with speech separation, speaker diarization, and automatic speech recognition (ASR) in the last decade, it has become possible to build pipelines that achieve reasonable error rates on this task. In this paper, we propose an end-to-end modular system for the LibriCSS meeting data, which combines independently trained separation, diarization, and recognition components, in that order. We study the effect of different state-of-the-art methods at each stage of the pipeline, and report results using task-specific metrics like SDR and DER, as well as downstream WER. Experiments indicate that the problem of overlapping speech for diarization and ASR can be effectively mitigated with the presence of a well-trained separation module. Our best system achieves a speaker-attributed WER of 12.7%, which is close to that of a non-overlapping ASR.