# Mitigating Speaker Leakage in Cascaded Multi-talker ASR with Diarization-based Transcript Correction

arXiv: https://arxiv.org/abs/2608.22196 · HTML: https://mlsanity.com/paper/2608.22196/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 26 references that are in the map.
- First posted on arXiv 23 Aug 2026.

## Builds on (most cited first, 21 of 21)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Attention is All You Need in Speech Separation](https://mlsanity.com/paper/2010.13154/) (2020)
- [WHAM!: Extending Speech Separation to Noisy Environments](https://mlsanity.com/paper/1907.01160/) (2019)
- [LibriMix: An Open-Source Dataset for Generalizable Speech Separation](https://mlsanity.com/paper/2005.11262/) (2020)
- [Serialized Output Training for End-to-End Overlapped Speech Recognition](https://mlsanity.com/paper/2003.12687/) (2020)
- [MossFormer2: Combining Transformer and RNN-Free Recurrent Network for Enhanced Time-Domain Monaural Speech Separation](https://mlsanity.com/paper/2312.11825/) (2023)
- [Continuous Speech Separation with Conformer](https://mlsanity.com/paper/2008.05773/) (2020)
- [Large-Scale Pre-Training of End-to-End Multi-Talker ASR for Meeting Transcription with Single Distant Microphone](https://mlsanity.com/paper/2103.16776/) (2021)
- [Turn-to-Diarize: Online Speaker Diarization Constrained by Transformer Transducer Speaker Turn Detection](https://mlsanity.com/paper/2109.11641/) (2021)
- [The Sound of Healthcare: Improving Medical Transcription ASR Accuracy with Large Language Models](https://mlsanity.com/paper/2402.07658/) (2024)
- [Enhancing Speaker Diarization with Large Language Models: A Contextual Beam Search Approach](https://mlsanity.com/paper/2309.05248/) (2023)
- [DiarizationLM: Speaker Diarization Post-Processing with Large Language Models](https://mlsanity.com/paper/2401.03506/) (2024)
- [Lexical Speaker Error Correction: Leveraging Language Models for Speaker Diarization Error Correction](https://mlsanity.com/paper/2306.09313/) (2023)
- [Anatomy of Industrial Scale Multilingual ASR](https://mlsanity.com/paper/2404.09841/) (2024)
- [Empowering Whisper as a Joint Multi-Talker and Target-Talker Speech Recognition System](https://mlsanity.com/paper/2407.09817/) (2024)
- [Speaker Targeting via Self-Speaker Adaptation for Multi-talker ASR](https://mlsanity.com/paper/2506.22646/) (2025)
- [Directed Speech Separation for Automatic Speech Recognition of Long Form Conversational Speech](https://mlsanity.com/paper/2112.05863/) (2021)
- [Serialized Output Prompting for Large Language Model-based Multi-Talker Speech Recognition](https://mlsanity.com/paper/2509.04488/) (2025)
- [AG-LSEC: Audio Grounded Lexical Speaker Error Correction](https://mlsanity.com/paper/2406.17266/) (2024)
- [SEAL: Speaker Error Correction using Acoustic-conditioned Large Language Models](https://mlsanity.com/paper/2501.08421/) (2025)
- [Error Analysis in a Modular Meeting Transcription System](https://mlsanity.com/paper/2509.10143/) (2025)

## Abstract (arXiv)

While cascaded multi-talker ASR (MT-ASR) leverages state-of-the-art foundation models, its performance is often capped by speaker leakage during separation. Prior correction strategies primarily focus on lexical re-labeling for speaker attribution. We propose a complementary pruning-based paradigm that robustly identifies and removes leakage artifacts. Our method utilizes a pre-trained speaker diarization model as a multimodal verifier to prune transcribed segments satisfying a tripartite consensus of temporal containment, lexical cross-validation, and temporal alignment. Results on LibriMix, LibriSpeechMix, and the AMI Meeting corpus show our algorithm consistently reduces cpW ER across diverse overlap conditions. Specifically, on subsets with high speaker leakage, our method achieves relative cpW ER reductions of up to 29%, highlighting its effectiveness in enhancing the reliability of cascaded MT-ASR transcripts in complex acoustic environments.