# Multi-class Spectral Clustering with Overlaps for Speaker Diarization

arXiv: https://arxiv.org/abs/2011.02900 · HTML: https://mlsanity.com/paper/2011.02900/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Nov 2020.

## Built on by (most cited first, 8 of 8)

- [One model to rule them all ? Towards End-to-End Joint Speaker Diarization and Speech Recognition](https://mlsanity.com/paper/2310.01688/) (2023)
- [In search of strong embedding extractors for speaker diarisation](https://mlsanity.com/paper/2210.14682/) (2022)
- [End-to-End Integration of Speech Separation and Voice Activity Detection for Low-Latency Diarization of Telephone Conversations](https://mlsanity.com/paper/2303.12002/) (2023)
- [TSUP Speaker Diarization System for Conversational Short-phrase Speaker Diarization Challenge](https://mlsanity.com/paper/2210.14653/) (2022)
- [A Teacher-Student approach for extracting informative speaker embeddings from speech mixtures](https://mlsanity.com/paper/2306.00634/) (2023)
- [Joint Speech Activity and Overlap Detection with Multi-Exit Architecture](https://mlsanity.com/paper/2209.11906/) (2022)
- [Towards Measuring and Scoring Speaker Diarization Fairness](https://mlsanity.com/paper/2302.09991/) (2023)
- [Geodesic interpolation of frame-wise speaker embeddings for the diarization of meeting scenarios](https://mlsanity.com/paper/2401.03963/) (2024)

## Abstract (arXiv)

This paper describes a method for overlap-aware speaker diarization. Given an overlap detector and a speaker embedding extractor, our method performs spectral clustering of segments informed by the output of the overlap detector. This is achieved by transforming the discrete clustering problem into a convex optimization problem which is solved by eigen-decomposition. Thereafter, we discretize the solution by alternatively using singular value decomposition and a modified version of non-maximal suppression which is constrained by the output of the overlap detector. Furthermore, we detail an HMM-DNN based overlap detector which performs frame-level classification and enforces duration constraints through HMM state transitions. Our method achieves a test diarization error rate (DER) of 24.0% on the mixed-headset setting of the AMI meeting corpus, which is a relative improvement of 15.2% over a strong agglomerative hierarchical clustering baseline, and compares favorably with other overlap-aware diarization methods. Further analysis on the LibriCSS data demonstrates the effectiveness of the proposed method in high overlap conditions.