# StarGANv2-VC: A Diverse, Unsupervised, Non-parallel Framework for Natural-Sounding Voice Conversion

arXiv: https://arxiv.org/abs/2107.10394 · HTML: https://mlsanity.com/paper/2107.10394/

## Key facts

- Cited by 58 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 29 references that are in the map.
- First posted on arXiv 21 Jul 2021.

## Builds on (most cited first, 20 of 20)

- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [StarGAN v2: Diverse Image Synthesis for Multiple Domains](https://mlsanity.com/paper/1912.01865/) (2019)
- [ESPnet: End-to-End Speech Processing Toolkit](https://mlsanity.com/paper/1804.00015/) (2018)
- [Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram](https://mlsanity.com/paper/1910.11480/) (2019)
- [Joint CTC-Attention based End-to-End Speech Recognition using Multi-task Learning](https://mlsanity.com/paper/1609.06773/) (2016)
- [AUTOVC: Zero-Shot Voice Style Transfer with Only Autoencoder Loss](https://mlsanity.com/paper/1905.05879/) (2019)
- [MOSNet: Deep Learning based Objective Assessment for Voice Conversion](https://mlsanity.com/paper/1904.08352/) (2019)
- [Seen and Unseen emotional style transfer for voice conversion with a new emotional speech dataset](https://mlsanity.com/paper/2010.14794/) (2020)
- [Voice Conversion Challenge 2020: Intra-lingual semi-parallel and cross-lingual voice conversion](https://mlsanity.com/paper/2008.12527/) (2020)
- [StarGAN-VC2: Rethinking Conditional Methods for StarGAN-Based Voice Conversion](https://mlsanity.com/paper/1907.12279/) (2019)
- [F0-consistent many-to-many non-parallel voice conversion via conditional autoencoder](https://mlsanity.com/paper/2004.07370/) (2020)
- [Voice Transformer Network: Sequence-to-Sequence Voice Conversion Using Transformer with Text-to-Speech Pretraining](https://mlsanity.com/paper/1912.06813/) (2019)
- [Multi-target Voice Conversion without Parallel Data by Adversarially Learning Disentangled Audio Representations](https://mlsanity.com/paper/1804.02812/) (2018)
- [CycleGAN-VC3: Examining and Improving CycleGAN-VCs for Mel-spectrogram Conversion](https://mlsanity.com/paper/2010.11672/) (2020)
- [Unsupervised Cross-Domain Singing Voice Conversion](https://mlsanity.com/paper/2008.02830/) (2020)
- [AttS2S-VC: Sequence-to-Sequence Voice Conversion with Attention and Context Preservation Mechanisms](https://mlsanity.com/paper/1811.04076/) (2018)
- [JVS corpus: free Japanese multi-speaker voice corpus](https://mlsanity.com/paper/1908.06248/) (2019)
- [Predictions of Subjective Ratings and Spoofing Assessments of Voice Conversion Challenge 2020 Submissions](https://mlsanity.com/paper/2009.03554/) (2020)
- [Cotatron: Transcription-Guided Speech Encoder for Any-to-Many Voice Conversion without Parallel Data](https://mlsanity.com/paper/2005.03295/) (2020)
- [Unsupervised Representation Disentanglement using Cross Domain Features and Adversarial Learning in Variational Autoencoder based Voice Conversion](https://mlsanity.com/paper/2001.07849/) (2020)

## Built on by (most cited first, 30 of 58)

- [StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Models](https://mlsanity.com/paper/2306.07691/) (2023)
- [ASVspoof 5: Crowdsourced Speech Data, Deepfakes, and Adversarial Attacks at Scale](https://mlsanity.com/paper/2408.08739/) (2024)
- [StyleTTS: A Style-Based Generative Model for Natural and Diverse Text-to-Speech Synthesis](https://mlsanity.com/paper/2205.15439/) (2022)
- [Towards Stealthy Backdoor Attacks against Speech Recognition via Elements of Sound](https://mlsanity.com/paper/2307.08208/) (2023)
- [Towards Building Text-To-Speech Systems for the Next Billion Users](https://mlsanity.com/paper/2211.09536/) (2022)
- [The Singing Voice Conversion Challenge 2023](https://mlsanity.com/paper/2306.14422/) (2023)
- [Phoneme-Level BERT for Enhanced Prosody of Text-to-Speech with Grapheme Predictions](https://mlsanity.com/paper/2301.08810/) (2023)
- [Zero-Shot Voice Conditioning for Denoising Diffusion TTS Models](https://mlsanity.com/paper/2206.02246/) (2022)
- [HiFTNet: A Fast High-Quality Neural Vocoder with Harmonic-plus-Noise Filter and Inverse Short Time Fourier Transform](https://mlsanity.com/paper/2309.09493/) (2023)
- [Towards Improved Zero-shot Voice Conversion with Conditional DSVAE](https://mlsanity.com/paper/2205.05227/) (2022)
- [Hierarchical Diffusion Models for Singing Voice Neural Vocoder](https://mlsanity.com/paper/2210.07508/) (2022)
- [Nonparallel Emotional Voice Conversion For Unseen Speaker-Emotion Pairs Using Dual Domain Adversarial Network & Virtual Domain Pairing](https://mlsanity.com/paper/2302.10536/) (2023)
- [VSVC: Backdoor attack against Keyword Spotting based on Voiceprint Selection and Voice Conversion](https://mlsanity.com/paper/2212.10103/) (2022)
- [Deepfake CAPTCHA: A Method for Preventing Fake Calls](https://mlsanity.com/paper/2301.03064/) (2023)
- [Style-Talker: Finetuning Audio Language Model and Style-Based Text-to-Speech Model for Fast Spoken Dialogue Generation](https://mlsanity.com/paper/2408.11849/) (2024)
- [FastVoiceGrad: One-step Diffusion-Based Voice Conversion with Adversarial Conditional Diffusion Distillation](https://mlsanity.com/paper/2409.02245/) (2024)
- [StyleTTS-ZS: Efficient High-Quality Zero-Shot Text-to-Speech Synthesis with Distilled Time-Varying Style Diffusion](https://mlsanity.com/paper/2409.10058/) (2024)
- [Robust One-Shot Singing Voice Conversion](https://mlsanity.com/paper/2210.11096/) (2022)
- [Towards General-Purpose Text-Instruction-Guided Voice Conversion](https://mlsanity.com/paper/2309.14324/) (2023)
- [Improving Generalization for AI-Synthesized Voice Detection](https://mlsanity.com/paper/2412.19279/) (2024)
- [Prosody-Adaptable Audio Codecs for Zero-Shot Voice Conversion via In-Context Learning](https://mlsanity.com/paper/2505.15402/) (2025)
- [SongBsAb: A Dual Prevention Approach against Singing Voice Conversion based Illegal Song Covers](https://mlsanity.com/paper/2401.17133/) (2024)
- [Easy, Interpretable, Effective: openSMILE for voice deepfake detection](https://mlsanity.com/paper/2408.15775/) (2024)
- [VoiceWukong: Benchmarking Deepfake Voice Detection](https://mlsanity.com/paper/2409.06348/) (2024)
- [An Exhaustive Evaluation of TTS- and VC-based Data Augmentation for ASR](https://mlsanity.com/paper/2503.08954/) (2025)
- [DualVC: Dual-mode Voice Conversion using Intra-model Knowledge Distillation and Hybrid Predictive Coding](https://mlsanity.com/paper/2305.12425/) (2023)
- [Discussion Paper: The Threat of Real Time Deepfakes](https://mlsanity.com/paper/2306.02487/) (2023)
- [Optimal Transport Maps are Good Voice Converters](https://mlsanity.com/paper/2411.02402/) (2024)
- [Conan: A Chunkwise Online Network for Zero-Shot Adaptive Voice Conversion](https://mlsanity.com/paper/2507.14534/) (2025)
- [HiFi-VC: High Quality ASR-Based Voice Conversion](https://mlsanity.com/paper/2203.16937/) (2022)

## Abstract (arXiv)

We present an unsupervised non-parallel many-to-many voice conversion (VC) method using a generative adversarial network (GAN) called StarGAN v2. Using a combination of adversarial source classifier loss and perceptual loss, our model significantly outperforms previous VC models. Although our model is trained only with 20 English speakers, it generalizes to a variety of voice conversion tasks, such as any-to-many, cross-lingual, and singing conversion. Using a style encoder, our framework can also convert plain reading speech into stylistic speech, such as emotional and falsetto speech. Subjective and objective evaluation experiments on a non-parallel many-to-many voice conversion task revealed that our model produces natural sounding voices, close to the sound quality of state-of-the-art text-to-speech (TTS) based voice conversion methods without the need for text labels. Moreover, our model is completely convolutional and with a faster-than-real-time vocoder such as Parallel WaveGAN can perform real-time voice conversion.