# Advancing the dimensionality reduction of speaker embeddings for speaker diarisation: disentangling noise and informing speech activity

arXiv: https://arxiv.org/abs/2110.03380 · HTML: https://mlsanity.com/paper/2110.03380/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 30 references that are in the map.
- First posted on arXiv 7 Oct 2021.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [VoxCeleb2: Deep Speaker Recognition](https://mlsanity.com/paper/1806.05622/) (2018)
- [VoxCeleb: a large-scale speaker identification dataset](https://mlsanity.com/paper/1706.08612/) (2017)
- [MLS: A Large-Scale Multilingual Dataset for Speech Research](https://mlsanity.com/paper/2012.03411/) (2020)
- [A Light CNN for Deep Face Representation with Noisy Labels](https://mlsanity.com/paper/1511.02683/) (2015)
- [Spot the conversation: speaker diarisation in the wild](https://mlsanity.com/paper/2007.01216/) (2020)
- [Speaker Diarization with LSTM](https://mlsanity.com/paper/1710.10468/) (2017)
- [End-to-End Speaker Diarization for an Unknown Number of Speakers with Encoder-Decoder Based Attractors](https://mlsanity.com/paper/2005.09921/) (2020)
- [The Third DIHARD Diarization Challenge](https://mlsanity.com/paper/2012.01477/) (2020)
- [The Second DIHARD Diarization Challenge: Dataset, task, and baselines](https://mlsanity.com/paper/1906.07839/) (2019)
- [VoxSRC 2020: The Second VoxCeleb Speaker Recognition Challenge](https://mlsanity.com/paper/2012.06867/) (2020)
- [Advances in integration of end-to-end neural and clustering-based diarization for real conversational speech](https://mlsanity.com/paper/2105.09040/) (2021)
- [Speaker Diarization with Region Proposal Network](https://mlsanity.com/paper/2002.06220/) (2020)
- [Unsupervised Adversarial Invariance](https://mlsanity.com/paper/1809.10083/) (2018)
- [Delving into VoxCeleb: environment invariant speaker recognition](https://mlsanity.com/paper/1910.11238/) (2019)
- [Unsupervised Representation Disentanglement using Cross Domain Features and Adversarial Learning in Variational Autoencoder based Voice Conversion](https://mlsanity.com/paper/2001.07849/) (2020)
- [End-to-End Neural Diarization: Reformulating Speaker Diarization as Simple Multi-label Classification](https://mlsanity.com/paper/2003.02966/) (2020)
- [BUT System for the Second DIHARD Speech Diarization Challenge](https://mlsanity.com/paper/2002.11356/) (2020)
- [Look who's not talking](https://mlsanity.com/paper/2011.14885/) (2020)
- [BUT System Description for DIHARD Speech Diarization Challenge 2019](https://mlsanity.com/paper/1910.08847/) (2019)
- [Adapting Speaker Embeddings for Speaker Diarisation](https://mlsanity.com/paper/2104.02879/) (2021)
- [Online Streaming End-to-End Neural Diarization Handling Overlapping Speech and Flexible Numbers of Speakers](https://mlsanity.com/paper/2101.08473/) (2021)

## Abstract (arXiv)

The objective of this work is to train noise-robust speaker embeddings adapted for speaker diarisation. Speaker embeddings play a crucial role in the performance of diarisation systems, but they often capture spurious information such as noise, adversely affecting performance. Our previous work has proposed an auto-encoder-based dimensionality reduction module to help remove the redundant information. However, they do not explicitly separate such information and have also been found to be sensitive to hyper-parameter values. To this end, we propose two contributions to overcome these issues: (i) a novel dimensionality reduction framework that can disentangle spurious information from the speaker embeddings; (ii) the use of speech activity vector to prevent the speaker code from representing the background noise. Through a range of experiments conducted on four datasets, our approach consistently demonstrates the state-of-the-art performance among models without system fusion.