# Speakers Unembedded: Embedding-free Approach to Long-form Neural Diarization

arXiv: https://arxiv.org/abs/2406.18679 · HTML: https://mlsanity.com/paper/2406.18679/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 32 references that are in the map.
- First posted on arXiv 26 Jun 2024.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [MUSAN: A Music, Speech, and Noise Corpus](https://mlsanity.com/paper/1510.08484/) (2015)
- [Permutation Invariant Training of Deep Models for Speaker-Independent Multi-talker Speech Separation](https://mlsanity.com/paper/1607.00325/) (2016)
- [Powerset multi-class cross entropy loss for neural speaker diarization](https://mlsanity.com/paper/2310.13025/) (2023)
- [End-to-End Neural Speaker Diarization with Permutation-Free Objectives](https://mlsanity.com/paper/1909.05952/) (2019)
- [End-to-End Neural Speaker Diarization with Self-attention](https://mlsanity.com/paper/1909.06247/) (2019)
- [Speaker Diarization with LSTM](https://mlsanity.com/paper/1710.10468/) (2017)
- [End-to-End Speaker Diarization for an Unknown Number of Speakers with Encoder-Decoder Based Attractors](https://mlsanity.com/paper/2005.09921/) (2020)
- [Integrating end-to-end neural and clustering-based diarization: Getting the best of both worlds](https://mlsanity.com/paper/2010.13366/) (2020)
- [Advances in integration of end-to-end neural and clustering-based diarization for real conversational speech](https://mlsanity.com/paper/2105.09040/) (2021)
- [Towards Neural Diarization for Unlimited Numbers of Speakers Using Global and Local Attractors](https://mlsanity.com/paper/2107.01545/) (2021)
- [Auxiliary Loss of Transformer with Residual Connection for End-to-End Speaker Diarization](https://mlsanity.com/paper/2110.07116/) (2021)
- [DiaPer: End-to-End Neural Diarization with Perceiver-Based Attractors](https://mlsanity.com/paper/2312.04324/) (2023)
- [DIVE: End-to-end Speech Diarization via Iterative Speaker Embedding](https://mlsanity.com/paper/2105.13802/) (2021)
- [Attention-based Encoder-Decoder Network for End-to-End Neural Speaker Diarization with Target Speaker Attractor](https://mlsanity.com/paper/2305.10704/) (2023)
- [Neural Diarization with Non-autoregressive Intermediate Attractors](https://mlsanity.com/paper/2303.06806/) (2023)
- [Privacy-oriented manipulation of speaker representations](https://mlsanity.com/paper/2310.06652/) (2023)
- [Transformer Attractors for Robust and Efficient End-to-End Neural Diarization](https://mlsanity.com/paper/2312.06253/) (2023)
- [Robust End-to-End Diarization with Domain Adaptive Training and Multi-Task Learning](https://mlsanity.com/paper/2312.07136/) (2023)

## Built on by (most cited first, 1 of 1)

- [SEAL: Speaker Error Correction using Acoustic-conditioned Large Language Models](https://mlsanity.com/paper/2501.08421/) (2025)

## Abstract (arXiv)

End-to-end neural diarization (EEND) models offer significant improvements over traditional embedding-based Speaker Diarization (SD) approaches but falls short on generalizing to long-form audio with large number of speakers. EEND-vector-clustering method mitigates this by combining local EEND with global clustering of speaker embeddings from local windows, but this requires an additional speaker embedding framework alongside the EEND module. In this paper, we propose a novel framework applying EEND both locally and globally for long-form audio without separate speaker embeddings. This approach achieves significant relative DER reduction of 13% and 10% over the conventional 1-pass EEND on Callhome American English and RT03-CTS datasets respectively and marginal improvements over EEND-vector-clustering without the need for additional speaker embeddings. Furthermore, we discuss the computational complexity of our proposed framework and explore strategies for reducing processing times.