# Overlap-aware low-latency online speaker diarization based on end-to-end local segmentation

arXiv: https://arxiv.org/abs/2109.06483 · HTML: https://mlsanity.com/paper/2109.06483/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Sep 2021.

## Built on by (most cited first, 15 of 15)

- [Online Neural Diarization of Unlimited Numbers of Speakers Using Global and Local Attractors](https://mlsanity.com/paper/2206.02432/) (2022)
- [LLAMAPIE: Proactive In-Ear Conversation Assistants](https://mlsanity.com/paper/2505.04066/) (2025)
- [LiveFC: A System for Live Fact-Checking of Audio Streams](https://mlsanity.com/paper/2408.07448/) (2024)
- [Tight integration of neural- and clustering-based diarization through deep unfolding of infinite Gaussian mixture model](https://mlsanity.com/paper/2202.06524/) (2022)
- [End-to-End Integration of Speech Separation and Voice Activity Detection for Low-Latency Diarization of Telephone Conversations](https://mlsanity.com/paper/2303.12002/) (2023)
- [Online speaker diarization of meetings guided by speech separation](https://mlsanity.com/paper/2402.00067/) (2024)
- [Systematic Evaluation of Online Speaker Diarization Systems Regarding their Latency](https://mlsanity.com/paper/2407.04293/) (2024)
- [An approach to optimize inference of the DIART speaker diarization pipeline](https://mlsanity.com/paper/2408.02341/) (2024)
- [Probabilistic Fusion and Calibration of Neural Speaker Diarization Models](https://mlsanity.com/paper/2511.22696/) (2025)
- [Collar-aware Training for Streaming Speaker Change Detection in Broadcast Speech](https://mlsanity.com/paper/2205.07086/) (2022)
- [A Review of Common Online Speaker Diarization Methods](https://mlsanity.com/paper/2406.14464/) (2024)
- [Interactive In-Meeting Speaker Correction with Human Feedback](https://mlsanity.com/paper/2509.18377/) (2025)
- [O-EENC-SD: Efficient Online End-to-End Neural Clustering for Speaker Diarization](https://mlsanity.com/paper/2512.15229/) (2025)
- [Efficiency-Performance Trade-offs in Neural Speaker Diarization via Structured Pruning and Low-Bit Quantization](https://mlsanity.com/paper/2606.14030/) (2026)
- [FASTDIAR: Frame-level speaker encoder for Streaming Diarization](https://mlsanity.com/paper/2610.02941/) (2026)

## Abstract (arXiv)

We propose to address online speaker diarization as a combination of incremental clustering and local diarization applied to a rolling buffer updated every 500ms. Every single step of the proposed pipeline is designed to take full advantage of the strong ability of a recently proposed end-to-end overlap-aware segmentation to detect and separate overlapping speakers. In particular, we propose a modified version of the statistics pooling layer (initially introduced in the x-vector architecture) to give less weight to frames where the segmentation model predicts simultaneous speakers. Furthermore, we derive cannot-link constraints from the initial segmentation step to prevent two local speakers from being wrongfully merged during the incremental clustering step. Finally, we show how the latency of the proposed approach can be adjusted between 500ms and 5s to match the requirements of a particular use case, and we provide a systematic analysis of the influence of latency on the overall performance (on AMI, DIHARD and VoxConverse).