mlsanityData to 7 Oct 2026

An Overview of Voice Conversion and its Challenges: From Statistical Modeling to Deep Learning

arXiv 2008.03648 ↗First posted 9 Aug 2020eess.AS cs.SDcited by 111

Not on the map. Why?

Abstract (arXiv)

Speaker identity is one of the important characteristics of human speech. In voice conversion, we change the speaker identity from one to another, while keeping the linguistic content unchanged. Voice conversion involves multiple speech processing techniques, such as speech analysis, spectral conversion, prosody conversion, speaker characterization, and vocoding. With the recent advances in theory and practice, we are now able to produce human-like voice quality with high speaker similarity. In this paper, we provide a comprehensive overview of the state-of-the-art of voice conversion techniques and their performance evaluation methods from the statistical approaches to deep learning, and discuss their promise and limitations. We will also report the recent Voice Conversion Challenges (VCC), the performance of the current state of technology, and provide a summary of the available resources for voice conversion research.

Key facts. Cited by 111 papers in the map (data through 7 Oct 2026). First posted on arXiv 9 Aug 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 15; 2022: 18; 2023: 21; 2024: 20; 2025: 24; 2026: 13.

111 papers.

Sort:

By area:

  1. 2021A Survey on Neural Speech Synthesiscited by 182
  2. 2021Emotional Voice Conversion: Theory, Databases and ESDcited by 113
  3. 2021Deepfakes Generation and Detection: State-of-the-art, open challenges, countermeasures, and way forwardcited by 82
  4. 2023Voice Conversion With Just Nearest Neighborscited by 67
  5. 2022Emotion Intensity and its Control for Emotional Voice Conversioncited by 35
  6. 2022An Overview of Affective Speech Synthesis and Conversion in the Deep Learning Eracited by 32
  7. 2022Robust Disentangled Variational Speech Representation Learning for Zero-shot Voice Conversioncited by 28
  8. 2022Speech Synthesis with Mixed Emotionscited by 28
  9. 2023The Singing Voice Conversion Challenge 2023cited by 25
  10. 2022WaveFit: An Iterative and Non-autoregressive Neural Vocoder based on Fixed-Point Iterationcited by 20
  11. 2024SafeEar: Content Privacy-Preserving Audio Deepfake Detectioncited by 20
  12. 2025Speech DF Arena: A Leaderboard for Speech DeepFake Detection Modelscited by 20
38 more
  1. 2021Reinforcement Learning for Emotional Text-to-Speech Synthesis with Improved Emotion Discriminabilitycited by 19
  2. 2023The Ethical Implications of Generative Audio Models: A Systematic Literature Reviewcited by 18
  3. 2023PromptVC: Flexible Stylistic Voice Conversion in Latent Space Driven by Natural Language Promptscited by 18
  4. 2021Limited Data Emotional Voice Conversion Leveraging Text-to-Speech: Two-stage Sequence-to-Sequence Trainingcited by 17
  5. 2022CampNet: Context-Aware Mask Prediction for End-to-End Text-Based Speech Editingcited by 15
  6. 2022Speech Representation Disentanglement with Adversarial Mutual Information Learning for One-shot Voice Conversioncited by 14
  7. 2021VisualTTS: TTS with Accurate Lip-Speech Synchronization for Automatic Voice Overcited by 13
  8. 2021More than Words: In-the-Wild Visually-Driven Prosody for Text-to-Speechcited by 12
  9. 2022Towards Improved Zero-shot Voice Conversion with Conditional DSVAEcited by 10
  10. 2023Single and Multi-Speaker Cloned Voice Detection: From Perceptual to Learned Featurescited by 10
  11. 2021NVC-Net: End-to-End Adversarial Voice Conversioncited by 9
  12. 2021Contrastive Fine-grained Class Clustering via Generative Adversarial Networkscited by 9
  13. 2022A Comparative Study of Self-supervised Speech Representation Based Voice Conversioncited by 9
  14. 2023Fake the Real: Backdoor Attack on Deep Speech Classification via Voice Conversioncited by 9
  15. 2023Face-Driven Zero-Shot Voice Conversion with Memory-based Face-Voice Alignmentcited by 8
  16. 2025GenVC: Self-Supervised Zero-Shot Voice Conversioncited by 8
  17. 2022Dispersed Pixel Perturbation-based Imperceptible Backdoor Trigger for Image Classifier Modelscited by 7
  18. 2023Can Authorship Attribution Models Distinguish Speakers in Speech Transcripts?cited by 7
  19. 2024Enhancing the Stability of LLM-based Speech Generation Systems through Self-Supervised Representationscited by 7
  20. 2024A Review of Human Emotion Synthesis Based on Generative Technologycited by 7
  21. 2021Zero-shot Voice Conversion via Self-supervised Prosody Representation Learningcited by 6
  22. 2023Towards General-Purpose Text-Instruction-Guided Voice Conversioncited by 6
  23. 2025Prosody-Adaptable Audio Codecs for Zero-Shot Voice Conversion via In-Context Learningcited by 6
  24. 2023Advancing Stuttering Detection via Data Augmentation, Class-Balanced Loss and Multi-Contextual Deep Learningcited by 5
  25. 2024Anonymising Elderly and Pathological Speech: Voice Conversion Using DDSP and Query-by-Examplecited by 5
  26. 2025An Exhaustive Evaluation of TTS- and VC-based Data Augmentation for ASRcited by 5
  27. 2021StarGAN-ZSVC: Towards Zero-Shot Voice Conversion in Low-Resource Contextscited by 4
  28. 2021Improving multi-speaker TTS prosody variance with a residual encoder and normalizing flowscited by 4
  29. 2023High-Quality Automatic Voice Over with Accurate Alignment: Supervision through Self-Supervised Discrete Speech Unitscited by 4
  30. 2023Evaluating Methods for Ground-Truth-Free Foreign Accent Conversioncited by 4
  31. 2023Voice Conversion for Stuttered Speech, Instruments, Unseen Languages and Textually Described Voicescited by 4
  32. 2025FaceSpeak: Expressive and High-Quality Speech Synthesis from Human Portraits of Different Stylescited by 4
  33. 2026Self Voice Conversion as an Attack against Neural Audio Watermarkingcited by 4
  34. 2026Do Compact SSL Backbones Matter for Audio Deepfake Detection? A Controlled Study with RAPTORcited by 4
  35. 2021An Adaptive Learning based Generative Adversarial Network for One-To-One Voice Conversioncited by 3
  36. 2022HiFi-VC: High Quality ASR-Based Voice Conversioncited by 3
  37. 2022Speak Like a Dog: Human to Non-human creature Voice Conversioncited by 3
  38. 2023UnifySpeech: A Unified Framework for Zero-shot Text-to-Speech and Voice Conversioncited by 3

50 most cited of 111 All 111 (JSON)