# Multi-Scale Speaker Diarization With Neural Affinity Score Fusion

arXiv: https://arxiv.org/abs/2011.10527 · HTML: https://mlsanity.com/paper/2011.10527/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Nov 2020.

## Built on by (most cited first, 10 of 10)

- [A Review of Speaker Diarization: Recent Advances with Deep Learning](https://mlsanity.com/paper/2101.09624/) (2021)
- [DiarizationLM: Speaker Diarization Post-Processing with Large Language Models](https://mlsanity.com/paper/2401.03506/) (2024)
- [Multi-scale Speaker Diarization with Dynamic Scale Weighting](https://mlsanity.com/paper/2203.15974/) (2022)
- [A Quality-Guided Mixture of Score-Fusion Experts Framework for Human Recognition](https://mlsanity.com/paper/2508.00053/) (2025)
- [Highly Efficient Real-Time Streaming and Fully On-Device Speaker Diarization with Multi-Stage Clustering](https://mlsanity.com/paper/2210.13690/) (2022)
- [Multi-scale speaker embedding-based graph attention networks for speaker diarisation](https://mlsanity.com/paper/2110.03361/) (2021)
- [FusionAgent: A Multimodal Agent with Dynamic Model Selection for Human Recognition](https://mlsanity.com/paper/2603.26908/) (2026)
- [High-resolution embedding extractor for speaker diarisation](https://mlsanity.com/paper/2211.04060/) (2022)
- [Training-Free Affinity Fusion of Neural and Embedding-Based Speaker Diarization](https://mlsanity.com/paper/2609.39162/) (2026)
- [SepRQ : Self-Supervised Speech Mixture Representation Learning via Mask-Free, Multi-Scale Source Separation](https://mlsanity.com/paper/2610.04690/) (2026)

## Abstract (arXiv)

Identifying the identity of the speaker of short segments in human dialogue has been considered one of the most challenging problems in speech signal processing. Speaker representations of short speech segments tend to be unreliable, resulting in poor fidelity of speaker representations in tasks requiring speaker recognition. In this paper, we propose an unconventional method that tackles the trade-off between temporal resolution and the quality of the speaker representations. To find a set of weights that balance the scores from multiple temporal scales of segments, a neural affinity score fusion model is presented. Using the CALLHOME dataset, we show that our proposed multi-scale segmentation and integration approach can achieve a state-of-the-art diarization performance.