# Attention-based Encoder-Decoder Network for End-to-End Neural Speaker Diarization with Target Speaker Attractor

arXiv: https://arxiv.org/abs/2305.10704 · HTML: https://mlsanity.com/paper/2305.10704/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 May 2023.

## Built on by (most cited first, 5 of 5)

- [Speakers Unembedded: Embedding-free Approach to Long-form Neural Diarization](https://mlsanity.com/paper/2406.18679/) (2024)
- [Flow-TSVAD: Target-Speaker Voice Activity Detection via Latent Flow Matching](https://mlsanity.com/paper/2409.04859/) (2024)
- [Probabilistic Fusion and Calibration of Neural Speaker Diarization Models](https://mlsanity.com/paper/2511.22696/) (2025)
- [Geodesic interpolation of frame-wise speaker embeddings for the diarization of meeting scenarios](https://mlsanity.com/paper/2401.03963/) (2024)
- [Leveraging Speaker Embeddings in End-to-End Neural Diarization for Two-Speaker Scenarios](https://mlsanity.com/paper/2407.01317/) (2024)

## Abstract (arXiv)

This paper proposes a novel Attention-based Encoder-Decoder network for End-to-End Neural speaker Diarization (AED-EEND). In AED-EEND system, we incorporate the target speaker enrollment information used in target speaker voice activity detection (TS-VAD) to calculate the attractor, which can mitigate the speaker permutation problem and facilitate easier model convergence. In the training process, we propose a teacher-forcing strategy to obtain the enrollment information using the ground-truth label. Furthermore, we propose three heuristic decoding methods to identify the enrollment area for each speaker during the evaluation process. Additionally, we enhance the attractor calculation network LSTM used in the end-to-end encoder-decoder based attractor calculation (EEND-EDA) system by incorporating an attention-based model. By utilizing such an attention-based attractor decoder, our proposed AED-EEND system outperforms both the EEND-EDA and TS-VAD systems with only 0.5s of enrollment data.