# SIG-VC: A Speaker Information Guided Zero-shot Voice Conversion System for Both Human Beings and Machines

arXiv: https://arxiv.org/abs/2111.03811 · HTML: https://mlsanity.com/paper/2111.03811/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 26 references that are in the map.
- First posted on arXiv 6 Nov 2021.

## Builds on (most cited first, 15 of 15)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017)
- [ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification](https://mlsanity.com/paper/2005.07143/) (2020)
- [MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis](https://mlsanity.com/paper/1910.06711/) (2019)
- [AUTOVC: Zero-Shot Voice Style Transfer with Only Autoencoder Loss](https://mlsanity.com/paper/1905.05879/) (2019)
- [One-shot Voice Conversion by Separating Speaker and Content Representations with Instance Normalization](https://mlsanity.com/paper/1904.05742/) (2019)
- [Exploring the Encoding Layer and Loss Function in End-to-End Speaker and Language Recognition System](https://mlsanity.com/paper/1804.05160/) (2018)
- [AGAIN-VC: A One-shot Voice Conversion using Activation Guidance and Adaptive Instance Normalization](https://mlsanity.com/paper/2011.00316/) (2020)
- [VQVC+: One-Shot Voice Conversion by Vector Quantization and U-Net architecture](https://mlsanity.com/paper/2006.04154/) (2020)
- [Improving Zero-shot Voice Style Transfer via Disentangled Representation Learning](https://mlsanity.com/paper/2103.09420/) (2021)
- [From Speaker Verification to Multispeaker Speech Synthesis, Deep Transfer with Feedback Constraint](https://mlsanity.com/paper/2005.04587/) (2020)
- [GAZEV: GAN-Based Zero-Shot Voice Conversion over Non-parallel Speech Corpus](https://mlsanity.com/paper/2010.12788/) (2020)
- [ConVoice: Real-Time Zero-Shot Voice Style Transfer with Convolutional Network](https://mlsanity.com/paper/2005.07815/) (2020)
- [Optimizing voice conversion network with cycle consistency loss of speaker identity](https://mlsanity.com/paper/2011.08548/) (2020)
- [Building Bilingual and Code-Switched Voice Conversion with Limited Training Data Using Embedding Consistency Loss](https://mlsanity.com/paper/2104.10832/) (2021)

## Built on by (most cited first, 5 of 5)

- [GenVC: Self-Supervised Zero-Shot Voice Conversion](https://mlsanity.com/paper/2502.04519/) (2025)
- [Reimagining Speech: A Scoping Review of Deep Learning-Powered Voice Conversion](https://mlsanity.com/paper/2311.08104/) (2023)
- [Finding My Voice: Generative Reconstruction of Disordered Speech for Automated Clinical Evaluation](https://mlsanity.com/paper/2509.19231/) (2025)
- [DAST: A Dual-Stream Voice Anonymization Attacker with Staged Training](https://mlsanity.com/paper/2603.12840/) (2026)
- [CoDiff-VC: A Codec-Assisted Diffusion Model for Zero-shot Voice Conversion](https://mlsanity.com/paper/2411.18918/) (2024)

## Abstract (arXiv)

Nowadays, as more and more systems achieve good performance in traditional voice conversion (VC) tasks, people's attention gradually turns to VC tasks under extreme conditions. In this paper, we propose a novel method for zero-shot voice conversion. We aim to obtain intermediate representations for speaker-content disentanglement of speech to better remove speaker information and get pure content information. Accordingly, our proposed framework contains a module that removes the speaker information from the acoustic feature of the source speaker. Moreover, speaker information control is added to our system to maintain the voice cloning performance. The proposed system is evaluated by subjective and objective metrics. Results show that our proposed system significantly reduces the trade-off problem in zero-shot voice conversion, while it also manages to have high spoofing power to the speaker verification system.