# Online End-to-End Neural Diarization with Speaker-Tracing Buffer

arXiv: https://arxiv.org/abs/2006.02616 · HTML: https://mlsanity.com/paper/2006.02616/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Jun 2020.

## Built on by (most cited first, 14 of 14)

- [A Review of Speaker Diarization: Recent Advances with Deep Learning](https://mlsanity.com/paper/2101.09624/) (2021)
- [Online Neural Diarization of Unlimited Numbers of Speakers Using Global and Local Attractors](https://mlsanity.com/paper/2206.02432/) (2022)
- [Configurable Privacy-Preserving Automatic Speech Recognition](https://mlsanity.com/paper/2104.00766/) (2021)
- [DIVE: End-to-end Speech Diarization via Iterative Speaker Embedding](https://mlsanity.com/paper/2105.13802/) (2021)
- [Tight integration of neural- and clustering-based diarization through deep unfolding of infinite Gaussian mixture model](https://mlsanity.com/paper/2202.06524/) (2022)
- [The Conversational Short-phrase Speaker Diarization (CSSD) Task: Dataset, Evaluation Metric and Baselines](https://mlsanity.com/paper/2208.08042/) (2022)
- [End-to-End Integration of Speech Separation and Voice Activity Detection for Low-Latency Diarization of Telephone Conversations](https://mlsanity.com/paper/2303.12002/) (2023)
- [TSUP Speaker Diarization System for Conversational Short-phrase Speaker Diarization Challenge](https://mlsanity.com/paper/2210.14653/) (2022)
- [Towards Word-Level End-to-End Neural Speaker Diarization with Auxiliary Network](https://mlsanity.com/paper/2309.08489/) (2023)
- [A Review of Common Online Speaker Diarization Methods](https://mlsanity.com/paper/2406.14464/) (2024)
- [Exploring Speaker Diarization with Mixture of Experts](https://mlsanity.com/paper/2506.14750/) (2025)
- [O-EENC-SD: Efficient Online End-to-End Neural Clustering for Speaker Diarization](https://mlsanity.com/paper/2512.15229/) (2025)
- [Efficiency-Performance Trade-offs in Neural Speaker Diarization via Structured Pruning and Low-Bit Quantization](https://mlsanity.com/paper/2606.14030/) (2026)
- [FASTDIAR: Frame-level speaker encoder for Streaming Diarization](https://mlsanity.com/paper/2610.02941/) (2026)

## Abstract (arXiv)

This paper proposes a novel online speaker diarization algorithm based on a fully supervised self-attention mechanism (SA-EEND). Online diarization inherently presents a speaker's permutation problem due to the possibility to assign speaker regions incorrectly across the recording. To circumvent this inconsistency, we proposed a speaker-tracing buffer mechanism that selects several input frames representing the speaker permutation information from previous chunks and stores them in a buffer. These buffered frames are stacked with the input frames in the current chunk and fed into a self-attention network. Our method ensures consistent diarization outputs across the buffer and the current chunk by checking the correlation between their corresponding outputs. Additionally, we trained SA-EEND with variable chunk-sizes to mitigate the mismatch between training and inference introduced by the speaker-tracing buffer mechanism. Experimental results, including online SA-EEND and variable chunk-size, achieved DERs of 12.54% for CALLHOME and 20.77% for CSJ with 1.4s actual latency.