# Advances in integration of end-to-end neural and clustering-based diarization for real conversational speech

arXiv: https://arxiv.org/abs/2105.09040 · HTML: https://mlsanity.com/paper/2105.09040/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 May 2021.

## Built on by (most cited first, 23 of 23)

- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [Powerset multi-class cross entropy loss for neural speaker diarization](https://mlsanity.com/paper/2310.13025/) (2023)
- [SpeakerLM: End-to-End Versatile Speaker Diarization and Recognition with Multimodal Large Language Models](https://mlsanity.com/paper/2508.06372/) (2025)
- [Towards Neural Diarization for Unlimited Numbers of Speakers Using Global and Local Attractors](https://mlsanity.com/paper/2107.01545/) (2021)
- [One model to rule them all ? Towards End-to-End Joint Speaker Diarization and Speech Recognition](https://mlsanity.com/paper/2310.01688/) (2023)
- [Online Neural Diarization of Unlimited Numbers of Speakers Using Global and Local Attractors](https://mlsanity.com/paper/2206.02432/) (2022)
- [Target Speaker Voice Activity Detection with Transformers and Its Integration with End-to-End Neural Diarization](https://mlsanity.com/paper/2208.13085/) (2022)
- [Efficient and Robust Speaker Diarization via Structured Pruning of Self-Supervised Models](https://mlsanity.com/paper/2506.18623/) (2025)
- [Exploring Speech Foundation Models for Speaker Diarization in Child-Adult Dyadic Interactions](https://mlsanity.com/paper/2406.07890/) (2024)
- [Tight integration of neural- and clustering-based diarization through deep unfolding of infinite Gaussian mixture model](https://mlsanity.com/paper/2202.06524/) (2022)
- [Reformulating Speaker Diarization as Community Detection With Emphasis On Topological Structure](https://mlsanity.com/paper/2204.12112/) (2022)
- [BER: Balanced Error Rate For Speaker Diarization](https://mlsanity.com/paper/2211.04304/) (2022)
- [End-to-End Integration of Speech Separation and Voice Activity Detection for Low-Latency Diarization of Telephone Conversations](https://mlsanity.com/paper/2303.12002/) (2023)
- [Online speaker diarization of meetings guided by speech separation](https://mlsanity.com/paper/2402.00067/) (2024)
- [Summary of the NOTSOFAR-1 Challenge: Highlights and Learnings](https://mlsanity.com/paper/2501.17304/) (2025)
- [Recent Trends in Distant Conversational Speech Recognition: A Review of CHiME-7 and 8 DASR Challenges](https://mlsanity.com/paper/2507.18161/) (2025)
- [Benchmarking Diarization Models](https://mlsanity.com/paper/2509.26177/) (2025)
- [TOLD: A Novel Two-Stage Overlap-Aware Framework for Speaker Diarization](https://mlsanity.com/paper/2303.05397/) (2023)
- [Speakers Unembedded: Embedding-free Approach to Long-form Neural Diarization](https://mlsanity.com/paper/2406.18679/) (2024)
- [Advancing the dimensionality reduction of speaker embeddings for speaker diarisation: disentangling noise and informing speech activity](https://mlsanity.com/paper/2110.03380/) (2021)
- [Exploring Spoken Language Identification Strategies for Automatic Transcription of Multilingual Broadcast and Institutional Speech](https://mlsanity.com/paper/2406.09290/) (2024)
- [O-EENC-SD: Efficient Online End-to-End Neural Clustering for Speaker Diarization](https://mlsanity.com/paper/2512.15229/) (2025)
- [TCG CREST System Description for the DISPLACE-M Challenge](https://mlsanity.com/paper/2603.02030/) (2026)

## Abstract (arXiv)

Recently, we proposed a novel speaker diarization method called End-to-End-Neural-Diarization-vector clustering (EEND-vector clustering) that integrates clustering-based and end-to-end neural network-based diarization approaches into one framework. The proposed method combines advantages of both frameworks, i.e. high diarization performance and handling of overlapped speech based on EEND, and robust handling of long recordings with an arbitrary number of speakers based on clustering-based approaches. However, the method was only evaluated so far on simulated 2-speaker meeting-like data. This paper is to (1) report recent advances we made to this framework, including newly introduced robust constrained clustering algorithms, and (2) experimentally show that the method can now significantly outperform competitive diarization methods such as Encoder-Decoder Attractor (EDA)-EEND, on CALLHOME data which comprises real conversational speech data including overlapped speech and an arbitrary number of speakers. By further analyzing the experimental results, this paper also discusses pros and cons of the proposed method and reveals potential for further improvement. A set of the code to reproduce the results is available at https://github.com/nttcslab-sp/EEND-vector-clustering.