# Speaker diarization with session-level speaker embedding refinement using graph neural networks

arXiv: https://arxiv.org/abs/2005.11371 · HTML: https://mlsanity.com/paper/2005.11371/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 May 2020.

## Built on by (most cited first, 9 of 9)

- [A Review of Speaker Diarization: Recent Advances with Deep Learning](https://mlsanity.com/paper/2101.09624/) (2021)
- [Supervised Hierarchical Clustering using Graph Neural Networks for Speaker Diarization](https://mlsanity.com/paper/2302.12716/) (2023)
- [ATGNN: Audio Tagging Graph Neural Network](https://mlsanity.com/paper/2311.01526/) (2023)
- [Speaker attribution with voice profiles by graph-based semi-supervised learning](https://mlsanity.com/paper/2102.03634/) (2021)
- [End-to-End Supervised Hierarchical Graph Clustering for Speaker Diarization](https://mlsanity.com/paper/2401.12850/) (2024)
- [wav2graph: A Framework for Supervised Learning Knowledge Graph from Speech](https://mlsanity.com/paper/2408.04174/) (2024)
- [Speaker Diarization with Overlapping Community Detection Using Graph Attention Networks and Label Propagation Algorithm](https://mlsanity.com/paper/2506.02610/) (2025)
- [Low resource cross-modal alignment using HGNN to enhance speech representation](https://mlsanity.com/paper/2609.23191/) (2026)
- [Enhancing speech representation learning with cross-modal knowledge transfer with HGNN under low resource settings: the case study of Yemba](https://mlsanity.com/paper/2609.23194/) (2026)

## Abstract (arXiv)

Deep speaker embedding models have been commonly used as a building block for speaker diarization systems; however, the speaker embedding model is usually trained according to a global loss defined on the training data, which could be sub-optimal for distinguishing speakers locally in a specific meeting session. In this work we present the first use of graph neural networks (GNNs) for the speaker diarization problem, utilizing a GNN to refine speaker embeddings locally using the structural information between speech segments inside each session. The speaker embeddings extracted by a pre-trained model are remapped into a new embedding space, in which the different speakers within a single session are better separated. The model is trained for linkage prediction in a supervised manner by minimizing the difference between the affinity matrix constructed by the refined embeddings and the ground-truth adjacency matrix. Spectral clustering is then applied on top of the refined embeddings. We show that the clustering performance of the refined speaker embeddings outperforms the original embeddings significantly on both simulated and real meeting data, and our system achieves the state-of-the-art result on the NIST SRE 2000 CALLHOME database.