# Disentangled Speaker Representation Learning via Mutual Information Minimization

arXiv: https://arxiv.org/abs/2208.08012 · HTML: https://mlsanity.com/paper/2208.08012/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Aug 2022.

## Built on by (most cited first, 3 of 3)

- [Disentangling Voice and Content with Self-Supervision for Speaker Recognition](https://mlsanity.com/paper/2310.01128/) (2023)
- [LASPA: Language Agnostic Speaker Disentanglement with Prefix-Tuned Cross-Attention](https://mlsanity.com/paper/2506.02083/) (2025)
- [Getting More for Less: Using Weak Labels and AV-Mixup for Robust Audio-Visual Speaker Verification](https://mlsanity.com/paper/2309.07115/) (2023)

## Abstract (arXiv)

Domain mismatch problem caused by speaker-unrelated feature has been a major topic in speaker recognition. In this paper, we propose an explicit disentanglement framework to unravel speaker-relevant features from speaker-unrelated features via mutual information (MI) minimization. To achieve our goal of minimizing MI between speaker-related and speaker-unrelated features, we adopt a contrastive log-ratio upper bound (CLUB), which exploits the upper bound of MI. Our framework is constructed in a 3-stage structure. First, in the front-end encoder, input speech is encoded into shared initial embedding. Next, in the decoupling block, shared initial embedding is split into separate speaker-related and speaker-unrelated embeddings. Finally, disentanglement is conducted by MI minimization in the last stage. Experiments on Far-Field Speaker Verification Challenge 2022 (FFSVC2022) demonstrate that our proposed framework is effective for disentanglement. Also, to utilize domain-unknown datasets containing numerous speakers, we pre-trained the front-end encoder with VoxCeleb datasets. We then fine-tuned the speaker embedding model in the disentanglement framework with FFSVC 2022 dataset. The experimental results show that fine-tuning with a disentanglement framework on a existing pre-trained model is valid and can further improve performance.