# DeID-VC: Speaker De-identification via Zero-shot Pseudo Voice Conversion

arXiv: https://arxiv.org/abs/2209.04530 · HTML: https://mlsanity.com/paper/2209.04530/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 31 references that are in the map.
- First posted on arXiv 9 Sep 2022.

## Builds on (most cited first, 18 of 18)

- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis](https://mlsanity.com/paper/2010.05646/) (2020)
- [ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification](https://mlsanity.com/paper/2005.07143/) (2020)
- [ESPnet: End-to-End Speech Processing Toolkit](https://mlsanity.com/paper/1804.00015/) (2018)
- [SpeechBrain: A General-Purpose Speech Toolkit](https://mlsanity.com/paper/2106.04624/) (2021)
- [Generalized End-to-End Loss for Speaker Verification](https://mlsanity.com/paper/1710.10467/) (2017)
- [AUTOVC: Zero-Shot Voice Style Transfer with Only Autoencoder Loss](https://mlsanity.com/paper/1905.05879/) (2019)
- [Wing Loss for Robust Facial Landmark Localisation with Convolutional Neural Networks](https://mlsanity.com/paper/1711.06753/) (2017)
- [Introducing the VoicePrivacy Initiative](https://mlsanity.com/paper/2005.01387/) (2020)
- [The VoicePrivacy 2020 Challenge: Results and findings](https://mlsanity.com/paper/2109.00648/) (2021)
- [Speaker Anonymization Using X-vector and Neural Waveform Models](https://mlsanity.com/paper/1905.13561/) (2019)
- [Evaluating Voice Conversion-based Privacy Protection against Informed Attackers](https://mlsanity.com/paper/1911.03934/) (2019)
- [Design Choices for X-vector Based Speaker Anonymization](https://mlsanity.com/paper/2005.08601/) (2020)
- [Voice-Indistinguishability: Protecting Voiceprint in Privacy-Preserving Speech Data Release](https://mlsanity.com/paper/2004.07442/) (2020)
- [VoiceMask: Anonymize and Sanitize Voice Input on Mobile Devices](https://mlsanity.com/paper/1711.11460/) (2017)
- [VAE-based Domain Adaptation for Speaker Verification](https://mlsanity.com/paper/1908.10092/) (2019)
- [VAE-based regularization for deep speaker embedding](https://mlsanity.com/paper/1904.03617/) (2019)

## Built on by (most cited first, 1 of 1)

- [Diff-HierVC: Diffusion-based Hierarchical Voice Conversion with Robust Pitch Generation and Masked Prior for Zero-shot Speaker Adaptation](https://mlsanity.com/paper/2311.04693/) (2023)

## Abstract (arXiv)

The widespread adoption of speech-based online services raises security and privacy concerns regarding the data that they use and share. If the data were compromised, attackers could exploit user speech to bypass speaker verification systems or even impersonate users. To mitigate this, we propose DeID-VC, a speaker de-identification system that converts a real speaker to pseudo speakers, thus removing or obfuscating the speaker-dependent attributes from a spoken voice. The key components of DeID-VC include a Variational Autoencoder (VAE) based Pseudo Speaker Generator (PSG) and a voice conversion Autoencoder (AE) under zero-shot settings. With the help of PSG, DeID-VC can assign unique pseudo speakers at speaker level or even at utterance level. Also, two novel learning objectives are added to bridge the gap between training and inference of zero-shot voice conversion. We present our experimental results with word error rate (WER) and equal error rate (EER), along with three subjective metrics to evaluate the generated output of DeID-VC. The result shows that our method substantially improved intelligibility (WER 10% lower) and de-identification effectiveness (EER 5% higher) compared to our baseline. Code and listening demo: https://github.com/a43992899/DeID-VC