# End-to-End Supervised Hierarchical Graph Clustering for Speaker Diarization

arXiv: https://arxiv.org/abs/2401.12850 · HTML: https://mlsanity.com/paper/2401.12850/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 54 references that are in the map.
- First posted on arXiv 23 Jan 2024.

## Builds on (most cited first, 30 of 32)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [VoxCeleb2: Deep Speaker Recognition](https://mlsanity.com/paper/1806.05622/) (2018)
- [VoxCeleb: a large-scale speaker identification dataset](https://mlsanity.com/paper/1706.08612/) (2017)
- [pyannote.audio: neural building blocks for speaker diarization](https://mlsanity.com/paper/1911.01255/) (2019)
- [Powerset multi-class cross entropy loss for neural speaker diarization](https://mlsanity.com/paper/2310.13025/) (2023)
- [End-to-end speaker segmentation for overlap-aware resegmentation](https://mlsanity.com/paper/2104.04045/) (2021)
- [Spot the conversation: speaker diarisation in the wild](https://mlsanity.com/paper/2007.01216/) (2020)
- [End-to-End Neural Speaker Diarization with Self-attention](https://mlsanity.com/paper/1909.06247/) (2019)
- [Speaker Diarization with LSTM](https://mlsanity.com/paper/1710.10468/) (2017)
- [Bayesian HMM clustering of x-vector sequences (VBx) in speaker diarization: theory, implementation and analysis on standard tasks](https://mlsanity.com/paper/2012.14952/) (2020)
- [End-to-End Speaker Diarization for an Unknown Number of Speakers with Encoder-Decoder Based Attractors](https://mlsanity.com/paper/2005.09921/) (2020)
- [BUT System Description to VoxCeleb Speaker Recognition Challenge 2019](https://mlsanity.com/paper/1910.12592/) (2019)
- [The Third DIHARD Diarization Challenge](https://mlsanity.com/paper/2012.01477/) (2020)
- [ECAPA-TDNN Embeddings for Speaker Diarization](https://mlsanity.com/paper/2104.01466/) (2021)
- [Overlap-aware diarization: resegmentation using neural end-to-end overlapped speech detection](https://mlsanity.com/paper/1910.11646/) (2019)
- [Learning Hierarchical Graph Neural Networks for Image Clustering](https://mlsanity.com/paper/2107.01319/) (2021)
- [LSTM based Similarity Measurement with Spectral Clustering for Speaker Diarization](https://mlsanity.com/paper/1907.10393/) (2019)
- [Towards Neural Diarization for Unlimited Numbers of Speakers Using Global and Local Attractors](https://mlsanity.com/paper/2107.01545/) (2021)
- [Neural Speaker Diarization with Speaker-Wise Chain Rule](https://mlsanity.com/paper/2006.01796/) (2020)
- [GPU-accelerated Guided Source Separation for Meeting Transcription](https://mlsanity.com/paper/2212.05271/) (2022)
- [Speaker diarization with session-level speaker embedding refinement using graph neural networks](https://mlsanity.com/paper/2005.11371/) (2020)
- [Supervised Hierarchical Clustering using Graph Neural Networks for Speaker Diarization](https://mlsanity.com/paper/2302.12716/) (2023)
- [DISPLACE Challenge: DIarization of SPeaker and LAnguage in Conversational Environments](https://mlsanity.com/paper/2303.00830/) (2023)
- [DiaPer: End-to-End Neural Diarization with Perceiver-Based Attractors](https://mlsanity.com/paper/2312.04324/) (2023)
- [Self-Supervised Metric Learning With Graph Clustering For Speaker Diarization](https://mlsanity.com/paper/2109.06824/) (2021)
- [Graph Convolutional Network Based Semi-Supervised Learning on Multi-Speaker Meeting Data](https://mlsanity.com/paper/2204.11501/) (2022)
- [Summary of the DISPLACE Challenge 2023 - DIarization of SPeaker and LAnguage in Conversational Environments](https://mlsanity.com/paper/2311.12564/) (2023)
- [Designing an Effective Metric Learning Pipeline for Speaker Diarization](https://mlsanity.com/paper/1811.00183/) (2018)

## Built on by (most cited first, 1 of 1)

- [Taxonomy Tree Generation from Citation Graph](https://mlsanity.com/paper/2410.03761/) (2024)

## Abstract (arXiv)

Speaker diarization, the task of segmenting an audio recording based on speaker identity, constitutes an important speech pre-processing step for several downstream applications.The conventional approach to diarization involves multiple steps of embedding extraction and clustering, which are often optimized in an isolated fashion. While end-to-end diarization systems attempt to learn a single model for the task, they are often cumbersome to train and require large supervised datasets. In this paper, we propose an end-to-end supervised hierarchical clustering algorithm based on graph neural networks (GNN), called End-to-end Supervised HierARchical Clustering (E-SHARC). The embedding extractor is initialized using a pre-trained x-vector model while the GNN model is trained initially using the x-vector embeddings from the pre-trained model. Finally, the E-SHARC model uses the front-end mel-filterbank features as input and jointly optimizes the embedding extractor and the GNN clustering module, performing representation learning, metric learning, and clustering with end-to-end optimization. Further, with additional inputs from an external overlap detector, the E-SHARC approach is capable of predicting the speakers in the overlapping speech regions. The experimental evaluation on benchmark datasets like AMI, Voxconverse and DISPLACE, illustrates that the proposed E-SHARC framework provides competitive diarization results using graph based clustering methods.