# TS-SEP: Joint Diarization and Separation Conditioned on Estimated Speaker Embeddings

arXiv: https://arxiv.org/abs/2303.03849 · HTML: https://mlsanity.com/paper/2303.03849/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Mar 2023.

## Built on by (most cited first, 12 of 12)

- [MeetEval: A Toolkit for Computation of Word Error Rates for Meeting Transcription Systems](https://mlsanity.com/paper/2307.11394/) (2023)
- [SpeakerLM: End-to-End Versatile Speaker Diarization and Recognition with Multimodal Large Language Models](https://mlsanity.com/paper/2508.06372/) (2025)
- [One model to rule them all ? Towards End-to-End Joint Speaker Diarization and Speech Recognition](https://mlsanity.com/paper/2310.01688/) (2023)
- [Unifying Diarization, Separation, and ASR with Multi-Speaker Encoder](https://mlsanity.com/paper/2508.20474/) (2025)
- [Multiple Choice Learning for Efficient Speech Separation with Many Speakers](https://mlsanity.com/paper/2411.18497/) (2024)
- [Recent Trends in Distant Conversational Speech Recognition: A Review of CHiME-7 and 8 DASR Challenges](https://mlsanity.com/paper/2507.18161/) (2025)
- [Combining TF-GridNet and Mixture Encoder for Continuous Speech Separation for Meeting Transcription](https://mlsanity.com/paper/2309.08454/) (2023)
- [Error Analysis in a Modular Meeting Transcription System](https://mlsanity.com/paper/2509.10143/) (2025)
- [Mask-Weighted Spatial Likelihood Coding for Speaker-Independent Joint Localization and Mask Estimation](https://mlsanity.com/paper/2410.19595/) (2024)
- [Robust Target Speaker Diarization and Separation via Augmented Speaker Embedding Sampling](https://mlsanity.com/paper/2508.06393/) (2025)
- [Echo: A Joint-Embedding Predictive Architecture for Speaker Diarization and Speech Recognition in a Shared Latent Space](https://mlsanity.com/paper/2606.01909/) (2026)
- [Weakly Guided and Autoregressive Beamformer Parameterization for Generalizable Moving Speaker Extraction in Higher-Order Ambisonics](https://mlsanity.com/paper/2607.04471/) (2026)

## Abstract (arXiv)

Since diarization and source separation of meeting data are closely related tasks, we here propose an approach to perform the two objectives jointly. It builds upon the target-speaker voice activity detection (TS-VAD) diarization approach, which assumes that initial speaker embeddings are available. We replace the final combined speaker activity estimation network of TS-VAD with a network that produces speaker activity estimates at a time-frequency resolution. Those act as masks for source extraction, either via masking or via beamforming. The technique can be applied both for single-channel and multi-channel input and, in both cases, achieves a new state-of-the-art word error rate (WER) on the LibriCSS meeting data recognition task. We further compute speaker-aware and speaker-agnostic WERs to isolate the contribution of diarization errors to the overall WER performance.