# Evaluating Voice Conversion-based Privacy Protection against Informed Attackers

arXiv: https://arxiv.org/abs/1911.03934 · HTML: https://mlsanity.com/paper/1911.03934/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Nov 2019.

## Built on by (most cited first, 30 of 31)

- [The VoicePrivacy 2020 Challenge: Results and findings](https://mlsanity.com/paper/2109.00648/) (2021)
- [The VoicePrivacy 2024 Challenge Evaluation Plan](https://mlsanity.com/paper/2404.02677/) (2024)
- [The VoicePrivacy 2022 Challenge Evaluation Plan](https://mlsanity.com/paper/2203.12468/) (2022)
- [Differentially Private Speaker Anonymization](https://mlsanity.com/paper/2202.11823/) (2022)
- [The Impact of Speech Anonymization on Pathology and Its Limits](https://mlsanity.com/paper/2404.08064/) (2024)
- [Face-Driven Zero-Shot Voice Conversion with Memory-based Face-Voice Alignment](https://mlsanity.com/paper/2309.09470/) (2023)
- [The Third VoicePrivacy Challenge: Preserving Emotional Expressiveness and Linguistic Content in Voice Anonymization](https://mlsanity.com/paper/2601.11846/) (2026)
- [Configurable Privacy-Preserving Automatic Speech Recognition](https://mlsanity.com/paper/2104.00766/) (2021)
- [The VoicePrivacy 2020 Challenge Evaluation Plan](https://mlsanity.com/paper/2205.07123/) (2022)
- [SafeSpeech: Robust and Universal Voice Protection Against Malicious Speech Synthesis](https://mlsanity.com/paper/2504.09839/) (2025)
- [Protecting gender and identity with disentangled speech representations](https://mlsanity.com/paper/2104.11051/) (2021)
- [StarGAN-ZSVC: Towards Zero-Shot Voice Conversion in Low-Resource Contexts](https://mlsanity.com/paper/2106.00043/) (2021)
- [On the invertibility of a voice privacy system using embedding alignement](https://mlsanity.com/paper/2110.05431/) (2021)
- [Fantomas: Understanding Face Anonymization Reversibility](https://mlsanity.com/paper/2210.10651/) (2022)
- [Introducing Model Inversion Attacks on Automatic Speaker Recognition](https://mlsanity.com/paper/2301.03206/) (2023)
- [The First VoicePrivacy Attacker Challenge](https://mlsanity.com/paper/2504.14183/) (2025)
- [Prosody-Driven Privacy-Preserving Dementia Detection](https://mlsanity.com/paper/2407.03470/) (2024)
- [Zero-Shot Voice Conversion via Content-Aware Timbre Ensemble and Conditional Flow Matching](https://mlsanity.com/paper/2411.02026/) (2024)
- [Safeguarding Privacy in Edge Speech Understanding with Tiny Foundation Models](https://mlsanity.com/paper/2502.01649/) (2025)
- [You Are What You Say: Exploiting Linguistic Content for VoicePrivacy Attacks](https://mlsanity.com/paper/2506.09521/) (2025)
- [LatentVoiceGrad: Nonparallel Voice Conversion with Latent Diffusion/Flow-Matching Models](https://mlsanity.com/paper/2509.08379/) (2025)
- [Evaluating X-vector-based Speaker Anonymization under White-box Assessment](https://mlsanity.com/paper/2109.11946/) (2021)
- [DeID-VC: Speaker De-identification via Zero-shot Pseudo Voice Conversion](https://mlsanity.com/paper/2209.04530/) (2022)
- [On the Impact of Voice Anonymization on Speech Diagnostic Applications: a Case Study on COVID-19 Detection](https://mlsanity.com/paper/2304.02181/) (2023)
- [Evaluation of Speaker Anonymization on Emotional Speech](https://mlsanity.com/paper/2305.01759/) (2023)
- [The First VoicePrivacy Attacker Challenge Evaluation Plan](https://mlsanity.com/paper/2410.07428/) (2024)
- [SEF-MK: Speaker-Embedding-Free Voice Anonymization through Multi-k-means Quantization](https://mlsanity.com/paper/2508.07086/) (2025)
- [Beyond Voice Identity Conversion: Manipulating Voice Attributes by Adversarial Learning of Structured Disentangled Representations](https://mlsanity.com/paper/2107.12346/) (2021)
- [Privacy-Utility Balanced Voice De-Identification Using Adversarial Examples](https://mlsanity.com/paper/2211.05446/) (2022)
- [Perceptual implications of automatic anonymization in pathological speech](https://mlsanity.com/paper/2505.00409/) (2025)

## Abstract (arXiv)

Speech data conveys sensitive speaker attributes like identity or accent. With a small amount of found data, such attributes can be inferred and exploited for malicious purposes: voice cloning, spoofing, etc. Anonymization aims to make the data unlinkable, i.e., ensure that no utterance can be linked to its original speaker. In this paper, we investigate anonymization methods based on voice conversion. In contrast to prior work, we argue that various linkage attacks can be designed depending on the attackers' knowledge about the anonymization scheme. We compare two frequency warping-based conversion methods and a deep learning based method in three attack scenarios. The utility of converted speech is measured via the word error rate achieved by automatic speech recognition, while privacy protection is assessed by the increase in equal error rate achieved by state-of-the-art i-vector or x-vector based speaker verification. Our results show that voice conversion schemes are unable to effectively protect against an attacker that has extensive knowledge of the type of conversion and how it has been applied, but may provide some protection against less knowledgeable attackers.