# Designing an Effective Metric Learning Pipeline for Speaker Diarization

arXiv: https://arxiv.org/abs/1811.00183 · HTML: https://mlsanity.com/paper/1811.00183/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Nov 2018.

## Built on by (most cited first, 3 of 3)

- [Contrastive Separative Coding for Self-supervised Representation Learning](https://mlsanity.com/paper/2103.00816/) (2021)
- [Lexical semantics enhanced neural word embeddings](https://mlsanity.com/paper/2210.00754/) (2022)
- [End-to-End Supervised Hierarchical Graph Clustering for Speaker Diarization](https://mlsanity.com/paper/2401.12850/) (2024)

## Abstract (arXiv)

State-of-the-art speaker diarization systems utilize knowledge from external data, in the form of a pre-trained distance metric, to effectively determine relative speaker identities to unseen data. However, much of recent focus has been on choosing the appropriate feature extractor, ranging from pre-trained i-vectors to representations learned via different sequence modeling architectures (e.g. 1D-CNNs, LSTMs, attention models), while adopting off-the-shelf metric learning solutions. In this paper, we argue that, regardless of the feature extractor, it is crucial to carefully design a metric learning pipeline, namely the loss function, the sampling strategy and the discrimnative margin parameter, for building robust diarization systems. Furthermore, we propose to adopt a fine-grained validation process to obtain a comprehensive evaluation of the generalization power of metric learning pipelines. To this end, we measure diarization performance across different language speakers, and variations in the number of speakers in a recording. Using empirical studies, we provide interesting insights into the effectiveness of different design choices and make recommendations.