mlsanityData to 7 Oct 2026

StarGANv2-VC: A Diverse, Unsupervised, Non-parallel Framework for Natural-Sounding Voice Conversion

arXiv 2107.10394 ↗First posted 21 Jul 2021cs.SD cs.LG eess.AScited by 57

Not on the map. Why?

Abstract (arXiv)

We present an unsupervised non-parallel many-to-many voice conversion (VC) method using a generative adversarial network (GAN) called StarGAN v2. Using a combination of adversarial source classifier loss and perceptual loss, our model significantly outperforms previous VC models. Although our model is trained only with 20 English speakers, it generalizes to a variety of voice conversion tasks, such as any-to-many, cross-lingual, and singing conversion. Using a style encoder, our framework can also convert plain reading speech into stylistic speech, such as emotional and falsetto speech. Subjective and objective evaluation experiments on a non-parallel many-to-many voice conversion task revealed that our model produces natural sounding voices, close to the sound quality of state-of-the-art text-to-speech (TTS) based voice conversion methods without the need for text labels. Moreover, our model is completely convolutional and with a faster-than-real-time vocoder such as Parallel WaveGAN can perform real-time voice conversion.

Key facts. Cited by 57 papers in the map (data through 7 Oct 2026). Builds on 20 of its 29 references that are in the map. First posted on arXiv 21 Jul 2021.

Builds on

20 of 29 references.

References · 20

  1. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 1,998
  2. 2019StarGAN v2: Diverse Image Synthesis for Multiple Domainscited by 1,095
  3. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 554
  4. 2019Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogramcited by 337
  5. 2016Joint CTC-Attention based End-to-End Speech Recognition using Multi-task Learningcited by 254
  6. 2019AUTOVC: Zero-Shot Voice Style Transfer with Only Autoencoder Losscited by 189
  7. 2019MOSNet: Deep Learning based Objective Assessment for Voice Conversioncited by 126
  8. 2020Seen and Unseen emotional style transfer for voice conversion with a new emotional speech datasetcited by 119
  9. 2020Voice Conversion Challenge 2020: Intra-lingual semi-parallel and cross-lingual voice conversioncited by 75
  10. 2019StarGAN-VC2: Rethinking Conditional Methods for StarGAN-Based Voice Conversioncited by 37
  11. 2020F0-consistent many-to-many non-parallel voice conversion via conditional autoencodercited by 35
  12. 2019Voice Transformer Network: Sequence-to-Sequence Voice Conversion Using Transformer with Text-to-Speech Pretrainingcited by 29
8 more
  1. 2018Multi-target Voice Conversion without Parallel Data by Adversarially Learning Disentangled Audio Representationscited by 25
  2. 2020CycleGAN-VC3: Examining and Improving CycleGAN-VCs for Mel-spectrogram Conversioncited by 25
  3. 2020Unsupervised Cross-Domain Singing Voice Conversioncited by 23
  4. 2018AttS2S-VC: Sequence-to-Sequence Voice Conversion with Attention and Context Preservation Mechanismscited by 22
  5. 2019JVS corpus: free Japanese multi-speaker voice corpuscited by 20
  6. 2020Predictions of Subjective Ratings and Spoofing Assessments of Voice Conversion Challenge 2020 Submissionscited by 19
  7. 2020Cotatron: Transcription-Guided Speech Encoder for Any-to-Many Voice Conversion without Parallel Datacited by 14
  8. 2020Unsupervised Representation Disentanglement using Cross Domain Features and Adversarial Learning in Variational Autoencoder based Voice Conversioncited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 9; 2023: 14; 2024: 15; 2025: 9; 2026: 10.

57 papers.

Sort:

By area:

  1. 2023StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Modelscited by 148
  2. 2024ASVspoof 5: Crowdsourced Speech Data, Deepfakes, and Adversarial Attacks at Scalecited by 88
  3. 2022StyleTTS: A Style-Based Generative Model for Natural and Diverse Text-to-Speech Synthesiscited by 32
  4. 2023Towards Stealthy Backdoor Attacks against Speech Recognition via Elements of Soundcited by 28
  5. 2022Towards Building Text-To-Speech Systems for the Next Billion Userscited by 27
  6. 2023The Singing Voice Conversion Challenge 2023cited by 25
  7. 2023Phoneme-Level BERT for Enhanced Prosody of Text-to-Speech with Grapheme Predictionscited by 22
  8. 2022Zero-Shot Voice Conditioning for Denoising Diffusion TTS Modelscited by 20
  9. 2022Towards Improved Zero-shot Voice Conversion with Conditional DSVAEcited by 10
  10. 2022Hierarchical Diffusion Models for Singing Voice Neural Vocodercited by 10
  11. 2023HiFTNet: A Fast High-Quality Neural Vocoder with Harmonic-plus-Noise Filter and Inverse Short Time Fourier Transformcited by 10
  12. 2023Nonparallel Emotional Voice Conversion For Unseen Speaker-Emotion Pairs Using Dual Domain Adversarial Network & Virtual Domain Pairingcited by 9
38 more
  1. 2023Deepfake CAPTCHA: A Method for Preventing Fake Callscited by 7
  2. 2024Style-Talker: Finetuning Audio Language Model and Style-Based Text-to-Speech Model for Fast Spoken Dialogue Generationcited by 7
  3. 2024FastVoiceGrad: One-step Diffusion-Based Voice Conversion with Adversarial Conditional Diffusion Distillationcited by 7
  4. 2022VSVC: Backdoor attack against Keyword Spotting based on Voiceprint Selection and Voice Conversioncited by 6
  5. 2023Towards General-Purpose Text-Instruction-Guided Voice Conversioncited by 6
  6. 2024StyleTTS-ZS: Efficient High-Quality Zero-Shot Text-to-Speech Synthesis with Distilled Time-Varying Style Diffusioncited by 6
  7. 2024Improving Generalization for AI-Synthesized Voice Detectioncited by 6
  8. 2025Prosody-Adaptable Audio Codecs for Zero-Shot Voice Conversion via In-Context Learningcited by 6
  9. 2022Robust One-Shot Singing Voice Conversioncited by 5
  10. 2024SongBsAb: A Dual Prevention Approach against Singing Voice Conversion based Illegal Song Coverscited by 5
  11. 2024Easy, Interpretable, Effective: openSMILE for voice deepfake detectioncited by 5
  12. 2024VoiceWukong: Benchmarking Deepfake Voice Detectioncited by 5
  13. 2025An Exhaustive Evaluation of TTS- and VC-based Data Augmentation for ASRcited by 5
  14. 2023Discussion Paper: The Threat of Real Time Deepfakescited by 4
  15. 2024Optimal Transport Maps are Good Voice Converterscited by 4
  16. 2025Conan: A Chunkwise Online Network for Zero-Shot Adaptive Voice Conversioncited by 4
  17. 2022HiFi-VC: High Quality ASR-Based Voice Conversioncited by 3
  18. 2022Speak Like a Dog: Human to Non-human creature Voice Conversioncited by 3
  19. 2023SLMGAN: Exploiting Speech Language Model Representations for Unsupervised Zero-Shot Voice Conversion in GANscited by 3
  20. 2023Attention-based Interactive Disentangling Network for Instance-level Emotional Voice Conversioncited by 3
  21. 2024Disentangling segmental and prosodic factors to non-native speech comprehensibilitycited by 3
  22. 2024Channel-Aware Domain-Adaptive Generative Adversarial Network for Robust Speech Recognitioncited by 3
  23. 2026AT-ADD: All-Type Audio Deepfake Detection Challenge Evaluation Plancited by 3
  24. 2024Unsupervised Disentanglement of Content and Style via Variance-Invariance Constraintscited by 2
  25. 2024EmoReg: Directional Latent Vector Modeling for Emotional Intensity Regularization in Diffusion-based Voice Conversioncited by 2
  26. 2025Emotion Recognition and Generation: A Comprehensive Review of Face, Speech, and Text Modalitiescited by 2
  27. 2025Generative Adversarial Network based Voice Conversion: Techniques, Challenges, and Recent Advancementscited by 2
  28. 2025FasterVoiceGrad: Faster One-step Diffusion-Based Voice Conversion with Adversarial Diffusion Conversion Distillationcited by 2
  29. 2025Vocoder-Projected Feature Discriminatorcited by 2
  30. 2026X-VC: Zero-shot Streaming Voice Conversion in Codec Spacecited by 2
  31. 2023Cross-modal Face- and Voice-style Transfercited by 1
  32. 2023Iteratively Improving Speech Recognition and Voice Conversioncited by 1
  33. 2024Effective Noise-aware Data Simulation for Domain-adaptive Speech Enhancement Leveraging Dynamic Stochastic Perturbationcited by 1
  34. 2025kNN-SVC: Robust Zero-Shot Singing Voice Conversion with Additive Synthesis and Concatenation Smoothness Optimizationcited by 1
  35. 2023Automatic Speech Disentanglement for Voice Conversion using Rank Module and Speech Augmentationcited by 0
  36. 2024Pureformer-VC: Non-parallel One-Shot Voice Conversion with Pure Transformer Blocks and Triplet Discriminative Trainingcited by 0
  37. 2025HuLA: Prosody-Aware Anti-Spoofing with Multi-Task Learning for Expressive and Emotional Synthetic Speechcited by 0
  38. 2026Universal Robust Speech Adaptation for Cross-Domain Speech Recognition and Enhancementcited by 0

50 most cited of 57 All 57 (JSON)