Generative Adversarial Network based Voice Conversion: Techniques, Challenges, and Recent Advancements
Not on the map. Why?
Abstract (arXiv)
Voice conversion (VC) stands as a crucial research area in speech synthesis, enabling the transformation of a speaker's vocal characteristics to resemble another while preserving the linguistic content. This technology has broad applications, including automated movie dubbing, speech-to-singing conversion, and assistive devices for pathological speech rehabilitation. With the increasing demand for high-quality and natural-sounding synthetic voices, researchers have developed a wide range of VC techniques. Among these, generative adversarial network (GAN)-based approaches have drawn considerable attention for their powerful feature-mapping capabilities and potential to produce highly realistic speech. Despite notable advancements, challenges such as ensuring training stability, maintaining linguistic consistency, and achieving perceptual naturalness continue to hinder progress in GAN-based VC systems. This systematic review presents a comprehensive analysis of the voice conversion landscape, highlighting key techniques, key challenges, and the transformative impact of GANs in the field. The survey categorizes existing methods, examines technical obstacles, and critically evaluates recent developments in GAN-based VC. By consolidating and synthesizing research findings scattered across the literature, this review provides a structured understanding of the strengths and limitations of different approaches. The significance of this survey lies in its ability to guide future research by identifying existing gaps, proposing potential directions, and offering insights for building more robust and efficient VC systems. Overall, this work serves as an essential resource for researchers, developers, and practitioners aiming to advance the state-of-the-art (SOTA) in voice conversion technology.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 65 of its 136 references that are in the map. First posted on arXiv 27 Apr 2025.
Builds on
65 of 136 references.
References · 65
- 2017Attention Is All You Needcited by 60,361
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2016Gaussian Error Linear Units (GELUs)cited by 3,502
- 2020GLU Variants Improve Transformercited by 1,804
- 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,154
- 2017StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translationcited by 1,039
- 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 804
- 2015Autoencoding beyond pixels using a learned similarity metriccited by 407
- 2020A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospectscited by 397
- 2019MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesiscited by 354
- 2016A Learned Representation For Artistic Stylecited by 341
- 2019Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogramcited by 340
38 more
- 2023Google USM: Scaling Automatic Speech Recognition Beyond 100 Languagescited by 231
- 2023Cross-Entropy Loss Functions: Theoretical Analysis and Applicationscited by 210
- 2019AUTOVC: Zero-Shot Voice Style Transfer with Only Autoencoder Losscited by 192
- 2021A Survey on Neural Speech Synthesiscited by 184
- 2020Seen and Unseen emotional style transfer for voice conversion with a new emotional speech datasetcited by 121
- 2017Fader Networks: Manipulating Images by Sliding Attributescited by 118
- 2020An Overview of Voice Conversion and its Challenges: From Statistical Modeling to Deep Learningcited by 112
- 2018StarGAN-VC: Non-parallel many-to-many voice conversion with star generative adversarial networkscited by 78
- 2020Voice Conversion Challenge 2020: Intra-lingual semi-parallel and cross-lingual voice conversioncited by 75
- 2018Patch-Based Image Inpainting with Generative Adversarial Networkscited by 70
- 2021StarGANv2-VC: A Diverse, Unsupervised, Non-parallel Framework for Natural-Sounding Voice Conversioncited by 58
- 2019Region Normalization for Image Inpaintingcited by 54
- 2017Variational Approaches for Auto-Encoding Generative Adversarial Networkscited by 52
- 2023A Review of Deep Learning Techniques for Speech Processingcited by 52
- 2019CycleGAN-VC2: Improved CycleGAN-based Non-parallel Voice Conversioncited by 46
- 2017Voice Conversion from Unaligned Corpora using Variational Autoencoding Wasserstein Generative Adversarial Networkscited by 38
- 2019StarGAN-VC2: Rethinking Conditional Methods for StarGAN-Based Voice Conversioncited by 37
- 2022Neural Vocoder is All You Need for Speech Super-resolutioncited by 37
- 2020F0-consistent many-to-many non-parallel voice conversion via conditional autoencodercited by 35
- 2019Voice Transformer Network: Sequence-to-Sequence Voice Conversion Using Transformer with Text-to-Speech Pretrainingcited by 29
- 2022Speech Synthesis with Mixed Emotionscited by 28
- 2020CycleGAN-VC3: Examining and Improving CycleGAN-VCs for Mel-spectrogram Conversioncited by 25
- 2021Universal Neural Vocoding with Parallel WaveNetcited by 25
- 2022AVQVC: One-shot Voice Conversion by Vector Quantization with applying contrastive learningcited by 24
- 2019JVS corpus: free Japanese multi-speaker voice corpuscited by 22
- 2020Converting Anyone's Emotion: Towards Speaker-Independent Emotional Voice Conversioncited by 22
- 2021A survey on Variational Autoencoders from a GreenAI perspectivecited by 22
- 2021MaskCycleGAN-VC: Learning Non-parallel Voice Conversion with Filling in Framescited by 18
- 2020Image Fine-grained Inpaintingcited by 16
- 2021EasyCall corpus: a dysarthric speech datasetcited by 15
- 2022A survey on GANs for computer vision: Recent research, analysis and taxonomycited by 15
- 2019MelGAN-VC: Voice Conversion and Audio Style Transfer on arbitrarily long samples using Spectrogramscited by 12
- 2020PPG-based singing voice conversion with adversarial representation learningcited by 12
- 2020GAZEV: GAN-Based Zero-Shot Voice Conversion over Non-parallel Speech Corpuscited by 10
- 2021Expressive Voice Conversion: A Joint Framework for Speaker Identity and Emotional Style Transfercited by 10
- 2021Towards Identity Preserving Normal to Dysarthric Voice Conversioncited by 10
- 2023Nonparallel Emotional Voice Conversion For Unseen Speaker-Emotion Pairs Using Dual Domain Adversarial Network & Virtual Domain Pairingcited by 9
- 2021VocBench: A Neural Vocoder Benchmark for Speech Synthesiscited by 8
50 most cited of 65 All 65 (JSON)
Built on by
Citing papers per year: 2025: 2.
2 papers.