Toward accurate speaker inventories for online speaker diarization
On the map: Speech and audio language models → Speaker recognition and diarization
Abstract (arXiv)
An online speaker diarizer maintains a growing inventory of speakers, and applications read it directly: a live transcript enumerates them as they appear. Existing systems either fix the inventory's size in advance, so speakers beyond that size are merged, or grow it on distance alone, so noise and turn boundaries split speakers. Scored on speaker count over nine public datasets, both failures appear where DER reports neither: on VoxSRC-23 the fixed-capacity systems return 44-46% too few speakers and the unbounded tracker with the lowest DER 63% too many. We build on that embedding-based tracker and change only what triggers a registration. A candidate speaker pool keeps unregistered embeddings apart and registers a speaker only once one candidate has gathered enough mutually compatible embeddings; commit-gated label assignment holds an output while its candidate is undecided, so no speaker appears in the output before it is confirmed. Across the nine datasets the macro speaker-count error falls from 225% to 21% of the reference and macro DER from 17.59% to 15.95%, at a mean added latency of 0.109s over all outputs; on VoxSRC-23 the tracker obtains both the lowest DER and the most accurate speaker count of the systems compared.
Key facts. No citing papers in the map yet (data through 19 Oct 2026). Placed in Speaker recognition and diarization, one of 11 directions in Speech and audio language models. Builds on 15 of its 26 references that are in the map. First posted on arXiv 8 Oct 2026.
Builds on
15 of 26 references.
Closest ancestors · 9
Same direction, newest first
- 2026ReDimNet2: Scaling Speaker Verification via Time-Pooled Dimension Reshaping12 Mar
- 2025Streaming Sortformer: Speaker Cache-Based Online Speaker Diarization with Arrival-Time Ordering24 Jul
- 2024Sequence-to-Sequence Neural Diarization with Automatic Speaker Detection and Representation21 Nov
- 2024The VoxCeleb Speaker Recognition Challenge: A Retrospective27 Aug
- 2022Online Neural Diarization of Unlimited Numbers of Speakers Using Global and Local Attractors6 Jun
- 2021Low-Latency Online Speaker Diarization with Graph-Based Label Generation27 Nov
- 2021M2MeT: The ICASSP 2022 Multi-Channel Multi-Party Meeting Transcription Challenge14 Oct
- 2021Overlap-aware low-latency online speaker diarization based on end-to-end local segmentation14 Sep
- 2021A Review of Speaker Diarization: Recent Advances with Deep Learning24 Jan
Other references · 6
- 2020Spot the conversation: speaker diarisation in the wildcited by 128
- 2019The Second DIHARD Diarization Challenge: Dataset, task, and baselinescited by 68
- 2020Online End-to-End Neural Diarization with Speaker-Tracing Buffercited by 27
- 2026FireRedASR2S: A State-of-the-Art Industrial-Grade All-in-One Automatic Speech Recognition Systemcited by 23
- 2022Absolute decision corrupts absolutely: conservative online speaker diarisationcited by 4
- 2024LS-EEND: Long-Form Streaming End-to-End Neural Diarization with Online Attractor Extractioncited by 4
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Beyond DER: Speaker Counting in Crowded End-to-End Diarization
- 2026Fast and Robust On-Device Speaker Diarization: Relative Minimum Cluster Size for Stride-Accelerated Pipelines
- 2022In search of strong embedding extractors for speaker diarisation
- 2026Speaker Tracking: Segment-online Multi-talker Organization with a Varying Number of Speakers ↗
- 2021Adapting Speaker Embeddings for Speaker Diarisation
- 2025Benchmarking Diarization Models
- 2026voxmap-studio: An open-source speaker diarization annotation tool with built-in cost instrumentation
- 2025Pushing the Limits of End-to-End Diarization