# Interpreting hierarchical organisation of speaker embeddings

arXiv: https://arxiv.org/abs/2609.15203 · HTML: https://mlsanity.com/paper/2609.15203/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 23 references that are in the map.
- First posted on arXiv 14 Sep 2026.

## Builds on (most cited first, 10 of 10)

- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [VoxCeleb2: Deep Speaker Recognition](https://mlsanity.com/paper/1806.05622/) (2018)
- [VoxCeleb: a large-scale speaker identification dataset](https://mlsanity.com/paper/1706.08612/) (2017)
- [Closed-Form Factorization of Latent Semantics in GANs](https://mlsanity.com/paper/2007.06600/) (2020)
- [Grad-CAM: Why did you say that?](https://mlsanity.com/paper/1611.07450/) (2016)
- [In defence of metric learning for speaker recognition](https://mlsanity.com/paper/2003.11982/) (2020)
- [Supervised Hierarchical Clustering using Graph Neural Networks for Speaker Diarization](https://mlsanity.com/paper/2302.12716/) (2023)
- [Intraclass clustering: an implicit learning ability that regularizes DNNs](https://mlsanity.com/paper/2103.06733/) (2021)
- [Explainable AI in Speaker Recognition -- Making Latent Representations Understandable](https://mlsanity.com/paper/2604.23354/) (2026)
- [Explainable AI in Speaker Recognition -- Attention Map Visualisation and Evaluation](https://mlsanity.com/paper/2606.22901/) (2026)

## Abstract (arXiv)

Speaker recognition neural networks recognise speaker identities from input utterances by learning latent representations (i.e. speaker embeddings). However, these networks' internal mechanisms remain largely opaque, motivating research in explainable artificial intelligence (XAI) to understand them. Existing studies have analysed how speaker embeddings are organised, but rarely frame these analyses within XAI. This work proposes to explain and interpret the organisation of speaker embeddings from an XAI perspective. To this end, we apply a hierarchical clustering algorithm, Single-Linkage Clustering (SLINK), to analyse whether our prepared speaker embeddings naturally form clusters with hierarchical relationships. The resulting hierarchical organisation (i.e. hierarchical clusters) is evaluated using the Cluster-Class Matching (CCM) method. Moreover, we propose a new method, termed Hierarchical Cluster-Class Matching (HCCM), to identify which hierarchical clusters best match individual semantic classes like male and conjunctive semantic classes like UK & male, thereby interpreting the clusters using their matched classes. We quantify the matching degree with a new metric called the L-score, which makes imperfect matches diagnosable. HCCM's results reveal that the hierarchical clusters analysed by SLINK are well interpreted using classes related to speaker identity, gender, and nationality, showing the semantics inside the hierarchical organisation of the examined speaker embeddings.