# Speaker Anonymization Using X-vector and Neural Waveform Models

arXiv: https://arxiv.org/abs/1905.13561 · HTML: https://mlsanity.com/paper/1905.13561/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 May 2019.

## Built on by (most cited first, 30 of 41)

- [The VoicePrivacy 2020 Challenge: Results and findings](https://mlsanity.com/paper/2109.00648/) (2021)
- [The VoicePrivacy 2024 Challenge Evaluation Plan](https://mlsanity.com/paper/2404.02677/) (2024)
- [The VoicePrivacy 2022 Challenge Evaluation Plan](https://mlsanity.com/paper/2203.12468/) (2022)
- [Differentially Private Speaker Anonymization](https://mlsanity.com/paper/2202.11823/) (2022)
- [The Impact of Speech Anonymization on Pathology and Its Limits](https://mlsanity.com/paper/2404.08064/) (2024)
- [Are disentangled representations all you need to build speaker anonymization systems?](https://mlsanity.com/paper/2208.10497/) (2022)
- [Privacy versus Emotion Preservation Trade-offs in Emotion-Preserving Speaker Anonymization](https://mlsanity.com/paper/2409.03655/) (2024)
- [Learning to Generate Text in Arbitrary Writing Styles](https://mlsanity.com/paper/2312.17242/) (2023)
- [Mitigating Unauthorized Speech Synthesis for Voice Protection](https://mlsanity.com/paper/2410.20742/) (2024)
- [Can Authorship Attribution Models Distinguish Speakers in Speech Transcripts?](https://mlsanity.com/paper/2311.07564/) (2023)
- [A Benchmark for Multi-speaker Anonymization](https://mlsanity.com/paper/2407.05608/) (2024)
- [The VoicePrivacy 2020 Challenge Evaluation Plan](https://mlsanity.com/paper/2205.07123/) (2022)
- [Improving Accented Speech Recognition with Multi-Domain Training](https://mlsanity.com/paper/2303.07924/) (2023)
- [SafeSpeech: Robust and Universal Voice Protection Against Malicious Speech Synthesis](https://mlsanity.com/paper/2504.09839/) (2025)
- [Probing the Feasibility of Multilingual Speaker Anonymization](https://mlsanity.com/paper/2407.02937/) (2024)
- [Analysis of Speech Temporal Dynamics in the Context of Speaker Verification and Voice Anonymization](https://mlsanity.com/paper/2412.17164/) (2024)
- [On the invertibility of a voice privacy system using embedding alignement](https://mlsanity.com/paper/2110.05431/) (2021)
- [Exploiting Context-dependent Duration Features for Voice Anonymization Attack Systems](https://mlsanity.com/paper/2507.15214/) (2025)
- [Asynchronous Voice Anonymization Using Adversarial Perturbation On Speaker Embedding](https://mlsanity.com/paper/2406.08200/) (2024)
- [Exploring VQ-VAE with Prosody Parameters for Speaker Anonymization](https://mlsanity.com/paper/2409.15882/) (2024)
- [The Impact of Automatic Speech Transcription on Speaker Attribution](https://mlsanity.com/paper/2507.08660/) (2025)
- [Evaluating X-vector-based Speaker Anonymization under White-box Assessment](https://mlsanity.com/paper/2109.11946/) (2021)
- [DeID-VC: Speaker De-identification via Zero-shot Pseudo Voice Conversion](https://mlsanity.com/paper/2209.04530/) (2022)
- [On the Impact of Voice Anonymization on Speech Diagnostic Applications: a Case Study on COVID-19 Detection](https://mlsanity.com/paper/2304.02181/) (2023)
- [Evaluation of Speaker Anonymization on Emotional Speech](https://mlsanity.com/paper/2305.01759/) (2023)
- [Vocoder drift compensation by x-vector alignment in speaker anonymisation](https://mlsanity.com/paper/2307.08403/) (2023)
- [Defending Against Authorship Identification Attacks](https://mlsanity.com/paper/2310.01568/) (2023)
- [Controllable Generation of Artificial Speaker Embeddings through Discovery of Principal Directions](https://mlsanity.com/paper/2310.17502/) (2023)
- [Speech After Gender: A Trans-Feminine Perspective on Next Steps for Speech Science and Technology](https://mlsanity.com/paper/2407.07235/) (2024)
- [SEF-MK: Speaker-Embedding-Free Voice Anonymization through Multi-k-means Quantization](https://mlsanity.com/paper/2508.07086/) (2025)

## Abstract (arXiv)

The social media revolution has produced a plethora of web services to which users can easily upload and share multimedia documents. Despite the popularity and convenience of such services, the sharing of such inherently personal data, including speech data, raises obvious security and privacy concerns. In particular, a user's speech data may be acquired and used with speech synthesis systems to produce high-quality speech utterances which reflect the same user's speaker identity. These utterances may then be used to attack speaker verification systems. One solution to mitigate these concerns involves the concealing of speaker identities before the sharing of speech data. For this purpose, we present a new approach to speaker anonymization. The idea is to extract linguistic and speaker identity features from an utterance and then to use these with neural acoustic and waveform models to synthesize anonymized speech. The original speaker identity, in the form of timbre, is suppressed and replaced with that of an anonymous pseudo identity. The approach exploits state-of-the-art x-vector speaker representations. These are used to derive anonymized pseudo speaker identities through the combination of multiple, random speaker x-vectors. Experimental results show that the proposed approach is effective in concealing speaker identities. It increases the equal error rate of a speaker verification system while maintaining high quality, anonymized speech.