# Design Choices for X-vector Based Speaker Anonymization

arXiv: https://arxiv.org/abs/2005.08601 · HTML: https://mlsanity.com/paper/2005.08601/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 May 2020.

## Built on by (most cited first, 27 of 27)

- [The VoicePrivacy 2020 Challenge: Results and findings](https://mlsanity.com/paper/2109.00648/) (2021)
- [The VoicePrivacy 2024 Challenge Evaluation Plan](https://mlsanity.com/paper/2404.02677/) (2024)
- [Anonymizing Speech with Generative Adversarial Networks to Preserve Speaker Privacy](https://mlsanity.com/paper/2210.07002/) (2022)
- [The VoicePrivacy 2022 Challenge Evaluation Plan](https://mlsanity.com/paper/2203.12468/) (2022)
- [Differentially Private Speaker Anonymization](https://mlsanity.com/paper/2202.11823/) (2022)
- [A Review of Speech-centric Trustworthy Machine Learning: Privacy, Safety, and Fairness](https://mlsanity.com/paper/2212.09006/) (2022)
- [Improving speaker de-identification with functional data analysis of f0 trajectories](https://mlsanity.com/paper/2203.16738/) (2022)
- [The Impact of Speech Anonymization on Pathology and Its Limits](https://mlsanity.com/paper/2404.08064/) (2024)
- [End-to-end streaming model for low-latency speech anonymization](https://mlsanity.com/paper/2406.09277/) (2024)
- [The Third VoicePrivacy Challenge: Preserving Emotional Expressiveness and Linguistic Content in Voice Anonymization](https://mlsanity.com/paper/2601.11846/) (2026)
- [Configurable Privacy-Preserving Automatic Speech Recognition](https://mlsanity.com/paper/2104.00766/) (2021)
- [A Benchmark for Multi-speaker Anonymization](https://mlsanity.com/paper/2407.05608/) (2024)
- [DarkStream: real-time speech anonymization with low latency](https://mlsanity.com/paper/2509.04667/) (2025)
- [Protecting gender and identity with disentangled speech representations](https://mlsanity.com/paper/2104.11051/) (2021)
- [The effect of speech pathology on automatic speaker verification -- a large-scale study](https://mlsanity.com/paper/2204.06450/) (2022)
- [Children's Voice Privacy: First Steps And Emerging Challenges](https://mlsanity.com/paper/2506.00100/) (2025)
- [TVTSyn: Content-Synchronous Time-Varying Timbre for Streaming Voice Conversion and Anonymization](https://mlsanity.com/paper/2602.09389/) (2026)
- [Introducing Model Inversion Attacks on Automatic Speaker Recognition](https://mlsanity.com/paper/2301.03206/) (2023)
- [Evaluating X-vector-based Speaker Anonymization under White-box Assessment](https://mlsanity.com/paper/2109.11946/) (2021)
- [DeID-VC: Speaker De-identification via Zero-shot Pseudo Voice Conversion](https://mlsanity.com/paper/2209.04530/) (2022)
- [On the Impact of Voice Anonymization on Speech Diagnostic Applications: a Case Study on COVID-19 Detection](https://mlsanity.com/paper/2304.02181/) (2023)
- [SEF-MK: Speaker-Embedding-Free Voice Anonymization through Multi-k-means Quantization](https://mlsanity.com/paper/2508.07086/) (2025)
- [StreamVoiceAnon+: Emotion-Preserving Streaming Speaker Anonymization via Frame-Level Acoustic Distillation](https://mlsanity.com/paper/2603.06079/) (2026)
- [PHONOS: PHOnetic Neutralization for Online Streaming Applications](https://mlsanity.com/paper/2603.27001/) (2026)
- [Anonymization, Not Elimination: Utility-Preserved Speech Anonymization](https://mlsanity.com/paper/2604.17000/) (2026)
- [Privacy-Utility Balanced Voice De-Identification Using Adversarial Examples](https://mlsanity.com/paper/2211.05446/) (2022)
- [Perceptual implications of automatic anonymization in pathological speech](https://mlsanity.com/paper/2505.00409/) (2025)

## Abstract (arXiv)

The recently proposed x-vector based anonymization scheme converts any input voice into that of a random pseudo-speaker. In this paper, we present a flexible pseudo-speaker selection technique as a baseline for the first VoicePrivacy Challenge. We explore several design choices for the distance metric between speakers, the region of x-vector space where the pseudo-speaker is picked, and gender selection. To assess the strength of anonymization achieved, we consider attackers using an x-vector based speaker verification system who may use original or anonymized speech for enrollment, depending on their knowledge of the anonymization scheme. The Equal Error Rate (EER) achieved by the attackers and the decoding Word Error Rate (WER) over anonymized data are reported as the measures of privacy and utility. Experiments are performed using datasets derived from LibriSpeech to find the optimal combination of design choices in terms of privacy and utility.