mlsanityData to 14 Oct 2026

One-shot Voice Conversion by Separating Speaker and Content Representations with Instance Normalization

arXiv 1904.05742 ↗First posted 10 Apr 2019cs.LG cs.SD eess.AScited by 82

Not on the map. Why?

Abstract (arXiv)

Recently, voice conversion (VC) without parallel data has been successfully adapted to multi-target scenario in which a single model is trained to convert the input voice to many different speakers. However, such model suffers from the limitation that it can only convert the voice to the speakers in the training data, which narrows down the applicable scenario of VC. In this paper, we proposed a novel one-shot VC approach which is able to perform VC by only an example utterance from source and target speaker respectively, and the source and target speaker do not even need to be seen during training. This is achieved by disentangling speaker and content representations with instance normalization (IN). Objective and subjective evaluation shows that our model is able to generate the voice similar to target speaker. In addition to the performance measurement, we also demonstrate that this model is able to learn meaningful speaker representations without any supervision.

Key facts. Cited by 82 papers in the map (data through 14 Oct 2026). First posted on arXiv 10 Apr 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 19; 2022: 20; 2023: 20; 2024: 8; 2025: 11; 2026: 4.

82 papers.

Sort:

By area:

  1. 2021Emotional Voice Conversion: Theory, Databases and ESDcited by 115
  2. 2021Diffusion-Based Voice Conversion with Fast Maximum Likelihood Sampling Schemecited by 100
  3. 2021Deepfakes Generation and Detection: State-of-the-art, open challenges, countermeasures, and way forwardcited by 82
  4. 2021Neural Analysis and Synthesis: Reconstructing Speech from Self-Supervised Representationscited by 81
  5. 2022ContentVec: An Improved Self-Supervised Speech Representation by Disentangling Speakerscited by 74
  6. 2023Voice Conversion With Just Nearest Neighborscited by 68
  7. 2021VQMIVC: Vector Quantization and Mutual Information-Based Unsupervised Speech Representation Disentanglement for One-shot Voice Conversioncited by 64
  8. 2021Investigating on Incorporating Pretrained and Learnable Speaker Representations for Multi-Speaker Multi-Style Text-to-Speechcited by 37
  9. 2022StyleTTS: A Style-Based Generative Model for Natural and Diverse Text-to-Speech Synthesiscited by 33
  10. 2022Robust Disentangled Variational Speech Representation Learning for Zero-shot Voice Conversioncited by 28
  11. 2023SEF-VC: Speaker Embedding Free Zero-Shot Voice Conversion with Cross Attentioncited by 19
  12. 2022Face-Dubbing++: Lip-Synchronous, Voice Preserving Translation of Videoscited by 18
38 more
  1. 2023Mimic: Speaking Style Disentanglement for Speech-Driven 3D Facial Animationcited by 16
  2. 2022Speaking Style Conversion in the Waveform Domain Using Discrete Self-Supervised Unitscited by 15
  3. 2023ACE-VC: Adaptive and Controllable Voice Conversion using Explicitly Disentangled Self-supervised Speech Representationscited by 15
  4. 2024StreamVC: Real-Time Low-Latency Voice Conversioncited by 15
  5. 2021Adversarially learning disentangled speech representations for robust multi-factor voice conversioncited by 14
  6. 2021Speaker Generationcited by 14
  7. 2022Speech Representation Disentanglement with Adversarial Mutual Information Learning for One-shot Voice Conversioncited by 14
  8. 2023Disentangling Voice and Content with Self-Supervision for Speaker Recognitioncited by 13
  9. 2022Disentangleing Content and Fine-grained Prosody Information via Hybrid ASR Bottleneck Features for Voice Conversioncited by 10
  10. 2022Towards Improved Zero-shot Voice Conversion with Conditional DSVAEcited by 10
  11. 2022Disentangled Feature Learning for Real-Time Neural Speech Codingcited by 10
  12. 2023Audio-Driven Dubbing for User Generated Contents via Style-Aware Semi-Parametric Synthesiscited by 10
  13. 2025SoK: How Robust is Audio Watermarking in Generative AI models?cited by 10
  14. 2021NVC-Net: End-to-End Adversarial Voice Conversioncited by 9
  15. 2022A Comparative Study of Self-supervised Speech Representation Based Voice Conversioncited by 9
  16. 2023Phoneme Hallucinator: One-shot Voice Conversion via Set Expansioncited by 9
  17. 2024DC-Spin: A Speaker-invariant Speech Tokenizer for Spoken Language Modelscited by 9
  18. 2021Training Robust Zero-Shot Voice Conversion Models with Self-supervised Featurescited by 7
  19. 2022Trusted Media Challenge Dataset and User Studycited by 7
  20. 2023Deepfake CAPTCHA: A Method for Preventing Fake Callscited by 7
  21. 2023SAIC: Integration of Speech Anonymization and Identity Classificationcited by 7
  22. 2024FastVoiceGrad: One-step Diffusion-Based Voice Conversion with Adversarial Conditional Diffusion Distillationcited by 7
  23. 2021Review of end-to-end speech synthesis technology based on deep learningcited by 6
  24. 2021Global Rhythm Style Transfer Without Text Transcriptionscited by 6
  25. 2021FMFCC-A: A Challenging Mandarin Dataset for Synthetic Speech Detectioncited by 6
  26. 2021Zero-shot Voice Conversion via Self-supervised Prosody Representation Learningcited by 6
  27. 2022Retriever: Learning Content-Style Representation as a Token-Level Bipartite Graphcited by 6
  28. 2022Robust One-Shot Singing Voice Conversioncited by 6
  29. 2023Towards General-Purpose Text-Instruction-Guided Voice Conversioncited by 6
  30. 2023CSLP-AE: A Contrastive Split-Latent Permutation Autoencoder Framework for Zero-Shot Electroencephalography Signal Conversioncited by 6
  31. 2025Prosody-Adaptable Audio Codecs for Zero-Shot Voice Conversion via In-Context Learningcited by 6
  32. 2021SIG-VC: A Speaker Information Guided Zero-shot Voice Conversion System for Both Human Beings and Machinescited by 5
  33. 2023SelfVC: Voice Conversion With Iterative Refinement using Self Transformationscited by 5
  34. 2023Self-Supervised Disentangled Representation Learning for Robust Target Speech Extractioncited by 5
  35. 2021NoiseVC: Towards High Quality Zero-Shot Voice Conversioncited by 4
  36. 2023Catch You and I Can: Revealing Source Voiceprint Against Voice Conversioncited by 4
  37. 2023Voice Conversion for Stuttered Speech, Instruments, Unseen Languages and Textually Described Voicescited by 4
  38. 2025RT-VC: Real-Time Zero-Shot Voice Conversion with Speech Articulatory Codingcited by 4

50 most cited of 82 All 82 (JSON)