# Causally Disentangled Contrastive Learning for Multilingual Speaker Embeddings

arXiv: https://arxiv.org/abs/2602.01363 · HTML: https://mlsanity.com/paper/2602.01363/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 53 references that are in the map.
- First posted on arXiv 1 Feb 2026.

## Builds on (most cited first, 20 of 20)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [Counterfactual Fairness](https://mlsanity.com/paper/1703.06856/) (2017)
- [Mitigating Unwanted Biases with Adversarial Learning](https://mlsanity.com/paper/1801.07593/) (2018)
- [ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification](https://mlsanity.com/paper/2005.07143/) (2020)
- [Censoring Representations with an Adversary](https://mlsanity.com/paper/1511.05897/) (2015)
- [Sentence Embedding Leaks More Information than You Expect: Generative Embedding Inversion Attack to Recover the Whole Sentence](https://mlsanity.com/paper/2305.03010/) (2023)
- [Bias in Automated Speaker Recognition](https://mlsanity.com/paper/2201.09486/) (2022)
- [Probing the Information Encoded in X-vectors](https://mlsanity.com/paper/1909.06351/) (2019)
- [Community-Based Fact-Checking on Twitter's Birdwatch Platform](https://mlsanity.com/paper/2104.07175/) (2021)
- [Improving fairness in speaker verification via Group-adapted Fusion Network](https://mlsanity.com/paper/2202.11323/) (2022)
- [Did the Roll-Out of Community Notes Reduce Engagement With Misinformation on X/Twitter?](https://mlsanity.com/paper/2307.07960/) (2023)
- [To train or not to train adversarially: A study of bias mitigation strategies for speaker recognition](https://mlsanity.com/paper/2203.09122/) (2022)
- [Who Checks the Checkers? Exploring Source Credibility in Twitter's Community Notes](https://mlsanity.com/paper/2406.12444/) (2024)
- [Demographic Attributes Prediction from Speech Using WavLM Embeddings](https://mlsanity.com/paper/2502.12007/) (2025)
- [WavShape: Information-Theoretic Speech Representation Learning for Fair and Privacy-Aware Audio Processing](https://mlsanity.com/paper/2506.22789/) (2025)
- [Fairness Metric Design Exploration in Multi-Domain Moral Sentiment Classification using Transformer-Based Models](https://mlsanity.com/paper/2510.11222/) (2025)
- [Assessing the quality and coherence of word embeddings after SCM-based intersectional bias mitigation](https://mlsanity.com/paper/2601.04393/) (2026)
- ["I don't trust them": Exploring Perceptions of Fact-checking Entities for Flagging Online Misinformation](https://mlsanity.com/paper/2410.00866/) (2024)

## Built on by (most cited first, 2 of 2)

- [L-Proto: Language-Aware Episodic Prototypical Training for Multilingual Speaker Verification](https://mlsanity.com/paper/2606.17416/) (2026)
- [Integrating Fairness and Explainability in a Multiple Instance Reinforcement Learning System](https://mlsanity.com/paper/2610.00035/) (2026)

## Abstract (arXiv)

Self-supervised speaker embeddings are widely used in speaker verification systems, but prior work has shown that they often encode sensitive demographic attributes, raising fairness and privacy concerns. This paper investigates the extent to which demographic information, specifically gender, age, and accent, is present in SimCLR-trained speaker embeddings and whether such leakage can be mitigated without severely degrading speaker verification performance. We study two debiasing strategies: adversarial training through gradient reversal and a causal bottleneck architecture that explicitly separates demographic and residual information. Demographic leakage is quantified using both linear and nonlinear probing classifiers, while speaker verification performance is evaluated using ROC-AUC and EER. Our results show that gender information is strongly and linearly encoded in baseline embeddings, whereas age and accent are weaker and primarily nonlinearly represented. Adversarial debiasing reduces gender leakage but has limited effect on age and accent and introduces a clear trade-off with verification accuracy. The causal bottleneck further suppresses demographic information, particularly in the residual representation, but incurs substantial performance degradation. These findings highlight fundamental limitations in mitigating demographic leakage in self-supervised speaker embeddings and clarify the trade-offs inherent in current debiasing approaches.