# X-Vectors with Multi-Scale Aggregation for Speaker Diarization

arXiv: https://arxiv.org/abs/2105.07367 · HTML: https://mlsanity.com/paper/2105.07367/

## Key facts

- No citing papers in the map yet (data through 19 Oct 2026).
- Placed in Speaker recognition and diarization, one of 11 directions in Speech and audio language models.
- Builds on 15 of its 31 references that are in the map.
- First posted on arXiv 16 May 2021.

## Placed in

- Speech and audio language models / [Speaker recognition and diarization](https://mlsanity.com/speech/speaker-recognition-and-diarization/)

## Builds on (most cited first, 15 of 15)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [VoxCeleb2: Deep Speaker Recognition](https://mlsanity.com/paper/1806.05622/) (2018)
- [VoxCeleb: a large-scale speaker identification dataset](https://mlsanity.com/paper/1706.08612/) (2017)
- [MUSAN: A Music, Speech, and Noise Corpus](https://mlsanity.com/paper/1510.08484/) (2015)
- [Generalized End-to-End Loss for Speaker Verification](https://mlsanity.com/paper/1710.10467/) (2017)
- [Speaker Diarization with LSTM](https://mlsanity.com/paper/1710.10468/) (2017)
- [Deep Speaker: an End-to-End Neural Speaker Embedding System](https://mlsanity.com/paper/1705.02304/) (2017)
- [Fully Supervised Speaker Diarization](https://mlsanity.com/paper/1810.04719/) (2018)
- [Auto-Tuning Spectral Clustering for Speaker Diarization Using Normalized Maximum Eigengap](https://mlsanity.com/paper/2003.02405/) (2020)
- [Deep Speaker Embedding Learning with Multi-Level Pooling for Text-Independent Speaker Verification](https://mlsanity.com/paper/1902.07821/) (2019)
- [Improving Multi-Scale Aggregation Using Feature Pyramid Module for Robust Speaker Verification of Variable-Duration Utterances](https://mlsanity.com/paper/2004.03194/) (2020)
- [Parallel training of DNNs with Natural Gradient and Parameter Averaging](https://mlsanity.com/paper/1410.7455/) (2014)
- [A Deep Neural Network for Short-Segment Speaker Recognition](https://mlsanity.com/paper/1907.10420/) (2019)
- [Triplet Network with Attention for Speaker Diarization](https://mlsanity.com/paper/1808.01535/) (2018)
- [Speaker Diarization using Deep Recurrent Convolutional Neural Networks for Speaker Embeddings](https://mlsanity.com/paper/1708.02840/) (2017)

## Abstract (arXiv)

Speaker diarization is the process of labeling different speakers in a speech signal. Deep speaker embeddings are generally extracted from short speech segments and clustered to determine the segments belong to same speaker identity. The x-vector, which embeds segment-level speaker characteristics by statistically pooling frame-level representations, is one of the most widely used deep speaker embeddings in speaker diarization. Multi-scale aggregation, which employs multi-scale representations from different layers, has recently successfully been used in short duration speaker verification. In this paper, we investigate a multi-scale aggregation approach in an x-vector embedding framework for speaker diarization by exploiting multiple statistics pooling layers from different frame-level layers. Thus, it is expected that x-vectors with multi-scale aggregation have the potential to capture meaningful speaker characteristics from short segments, effectively taking advantage of different information at multiple layers. Experimental evaluation on the CALLHOME dataset showed that our approach provides substantial improvement over the baseline x-vectors.