# Supervised Hierarchical Clustering using Graph Neural Networks for Speaker Diarization

arXiv: https://arxiv.org/abs/2302.12716 · HTML: https://mlsanity.com/paper/2302.12716/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 35 references that are in the map.
- First posted on arXiv 24 Feb 2023.

## Builds on (most cited first, 21 of 21)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [VoxCeleb2: Deep Speaker Recognition](https://mlsanity.com/paper/1806.05622/) (2018)
- [VoxCeleb: a large-scale speaker identification dataset](https://mlsanity.com/paper/1706.08612/) (2017)
- [Spot the conversation: speaker diarisation in the wild](https://mlsanity.com/paper/2007.01216/) (2020)
- [End-to-End Neural Speaker Diarization with Self-attention](https://mlsanity.com/paper/1909.06247/) (2019)
- [Speaker Diarization with LSTM](https://mlsanity.com/paper/1710.10468/) (2017)
- [Bayesian HMM clustering of x-vector sequences (VBx) in speaker diarization: theory, implementation and analysis on standard tasks](https://mlsanity.com/paper/2012.14952/) (2020)
- [End-to-End Speaker Diarization for an Unknown Number of Speakers with Encoder-Decoder Based Attractors](https://mlsanity.com/paper/2005.09921/) (2020)
- [BUT System Description to VoxCeleb Speaker Recognition Challenge 2019](https://mlsanity.com/paper/1910.12592/) (2019)
- [The Third DIHARD Diarization Challenge](https://mlsanity.com/paper/2012.01477/) (2020)
- [ECAPA-TDNN Embeddings for Speaker Diarization](https://mlsanity.com/paper/2104.01466/) (2021)
- [Learning Hierarchical Graph Neural Networks for Image Clustering](https://mlsanity.com/paper/2107.01319/) (2021)
- [LSTM based Similarity Measurement with Spectral Clustering for Speaker Diarization](https://mlsanity.com/paper/1907.10393/) (2019)
- [Towards Neural Diarization for Unlimited Numbers of Speakers Using Global and Local Attractors](https://mlsanity.com/paper/2107.01545/) (2021)
- [BUT System for the Second DIHARD Speech Diarization Challenge](https://mlsanity.com/paper/2002.11356/) (2020)
- [Analysis of the BUT Diarization System for VoxConverse Challenge](https://mlsanity.com/paper/2010.11718/) (2020)
- [Speaker diarization with session-level speaker embedding refinement using graph neural networks](https://mlsanity.com/paper/2005.11371/) (2020)
- [Self-Supervised Metric Learning With Graph Clustering For Speaker Diarization](https://mlsanity.com/paper/2109.06824/) (2021)
- [Graph Convolutional Network Based Semi-Supervised Learning on Multi-Speaker Meeting Data](https://mlsanity.com/paper/2204.11501/) (2022)
- [Self-supervised Representation Learning With Path Integral Clustering For Speaker Diarization](https://mlsanity.com/paper/2104.09456/) (2021)

## Built on by (most cited first, 7 of 7)

- [Adaptive Hierarchical Graph Cut for Multi-granularity Out-of-distribution Detection](https://mlsanity.com/paper/2412.15668/) (2024)
- [Explainable AI in Speaker Recognition -- Making Latent Representations Understandable](https://mlsanity.com/paper/2604.23354/) (2026)
- [End-to-End Supervised Hierarchical Graph Clustering for Speaker Diarization](https://mlsanity.com/paper/2401.12850/) (2024)
- [wav2graph: A Framework for Supervised Learning Knowledge Graph from Speech](https://mlsanity.com/paper/2408.04174/) (2024)
- [Emotion Recognition in Multi-Speaker Conversations through Speaker Identification, Knowledge Distillation, and Hierarchical Fusion](https://mlsanity.com/paper/2511.13731/) (2025)
- [Interpreting hierarchical organisation of speaker embeddings](https://mlsanity.com/paper/2609.15203/) (2026)
- [Low resource cross-modal alignment using HGNN to enhance speech representation](https://mlsanity.com/paper/2609.23191/) (2026)

## Abstract (arXiv)

Conventional methods for speaker diarization involve windowing an audio file into short segments to extract speaker embeddings, followed by an unsupervised clustering of the embeddings. This multi-step approach generates speaker assignments for each segment. In this paper, we propose a novel Supervised HierArchical gRaph Clustering algorithm (SHARC) for speaker diarization where we introduce a hierarchical structure using Graph Neural Network (GNN) to perform supervised clustering. The supervision allows the model to update the representations and directly improve the clustering performance, thus enabling a single-step approach for diarization. In the proposed work, the input segment embeddings are treated as nodes of a graph with the edge weights corresponding to the similarity scores between the nodes. We also propose an approach to jointly update the embedding extractor and the GNN model to perform end-to-end speaker diarization (E2E-SHARC). During inference, the hierarchical clustering is performed using node densities and edge existence probabilities to merge the segments until convergence. In the diarization experiments, we illustrate that the proposed E2E-SHARC approach achieves 53% and 44% relative improvements over the baseline systems on benchmark datasets like AMI and Voxconverse, respectively.