# Target-speaker Voice Activity Detection with Improved I-Vector Estimation for Unknown Number of Speaker

arXiv: https://arxiv.org/abs/2108.03342 · HTML: https://mlsanity.com/paper/2108.03342/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Aug 2021.

## Built on by (most cited first, 15 of 15)

- [Personal VAD 2.0: Optimizing Personal Voice Activity Detection for On-Device Speech Recognition](https://mlsanity.com/paper/2204.03793/) (2022)
- [Online Neural Diarization of Unlimited Numbers of Speakers Using Global and Local Attractors](https://mlsanity.com/paper/2206.02432/) (2022)
- [A Comparative Study on Speaker-attributed Automatic Speech Recognition in Multi-party Meetings](https://mlsanity.com/paper/2203.16834/) (2022)
- [Exploring Speaker-Related Information in Spoken Language Understanding for Better Speaker Diarization](https://mlsanity.com/paper/2305.12927/) (2023)
- [End-to-End Joint Target and Non-Target Speakers ASR](https://mlsanity.com/paper/2306.02273/) (2023)
- [LibriheavyMix: A 20,000-Hour Dataset for Single-Channel Reverberant Multi-Talker Speech Separation, ASR and Speaker Diarization](https://mlsanity.com/paper/2409.00819/) (2024)
- [Magnitude-aware Probabilistic Speaker Embeddings](https://mlsanity.com/paper/2202.13826/) (2022)
- [Neural Speaker Diarization Using Memory-Aware Multi-Speaker Embedding with Sequence-to-Sequence Architecture](https://mlsanity.com/paper/2309.09180/) (2023)
- [Self-supervised Pretraining for Robust Personalized Voice Activity Detection in Adverse Conditions](https://mlsanity.com/paper/2312.16613/) (2023)
- [SVVAD: Personal Voice Activity Detection for Speaker Verification](https://mlsanity.com/paper/2305.19581/) (2023)
- [Speaker Embedding-aware Neural Diarization for Flexible Number of Speakers with Textual Information](https://mlsanity.com/paper/2111.13694/) (2021)
- [Comparative Analysis of Personalized Voice Activity Detection Systems: Assessing Real-World Effectiveness](https://mlsanity.com/paper/2406.09443/) (2024)
- [The USTC-Ximalaya system for the ICASSP 2022 multi-channel multi-party meeting transcription (M2MeT) challenge](https://mlsanity.com/paper/2202.04855/) (2022)
- [Noise-Robust Target-Speaker Voice Activity Detection Through Self-Supervised Pretraining](https://mlsanity.com/paper/2501.03184/) (2025)
- [Robust Target Speaker Diarization and Separation via Augmented Speaker Embedding Sampling](https://mlsanity.com/paper/2508.06393/) (2025)

## Abstract (arXiv)

Target-speaker voice activity detection (TS-VAD) has recently shown promising results for speaker diarization on highly overlapped speech. However, the original model requires a fixed (and known) number of speakers, which limits its application to real conversations. In this paper, we extend TS-VAD to speaker diarization with unknown numbers of speakers. This is achieved by two steps: first, an initial diarization system is applied for speaker number estimation, followed by TS-VAD network output masking according to this estimate. We further investigate different diarization methods, including clustering-based and region proposal networks, for estimating the initial i-vectors. Since these systems have complementary strengths, we propose a fusion-based method to combine frame-level decisions from the systems for an improved initialization. We demonstrate through experiments on variants of the LibriCSS meeting corpus that our proposed approach can improve the DER by up to 50% relative across varying numbers of speakers. This improvement also results in better downstream ASR performance approaching that using oracle segments.