mlsanityData to 7 Oct 2026

FreeVC: Towards High-Quality Text-Free One-Shot Voice Conversion

arXiv 2210.15418 ↗First posted 27 Oct 2022cs.SD cs.LG eess.AScited by 79

Not on the map. Why?

Abstract (arXiv)

Voice conversion (VC) can be achieved by first extracting source content information and target speaker information, and then reconstructing waveform with these information. However, current approaches normally either extract dirty content information with speaker information leaked in, or demand a large amount of annotated data for training. Besides, the quality of reconstructed waveform can be degraded by the mismatch between conversion model and vocoder. In this paper, we adopt the end-to-end framework of VITS for high-quality waveform reconstruction, and propose strategies for clean content information extraction without text annotation. We disentangle content information by imposing an information bottleneck to WavLM features, and propose the spectrogram-resize based data augmentation to improve the purity of extracted content information. Experimental results show that the proposed method outperforms the latest VC models trained with annotated data and has greater robustness.

Key facts. Cited by 79 papers in the map (data through 7 Oct 2026). Builds on 25 of its 31 references that are in the map. First posted on arXiv 27 Oct 2022.

Builds on

25 of 31 references.

References · 25

  1. 2016Density estimation using Real NVPcited by 2,029
  2. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  3. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  4. 2016Least Squares Generative Adversarial Networkscited by 1,000
  5. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 639
  6. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 555
  7. 2020Unsupervised Cross-lingual Representation Learning for Speech Recognitioncited by 496
  8. 2015Autoencoding beyond pixels using a learned similarity metriccited by 407
  9. 2021YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyonecited by 277
  10. 2018Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesiscited by 257
  11. 2019AUTOVC: Zero-Shot Voice Style Transfer with Only Autoencoder Losscited by 189
  12. 2020Seen and Unseen emotional style transfer for voice conversion with a new emotional speech datasetcited by 119
13 more
  1. 2021Neural Analysis and Synthesis: Reconstructing Speech from Self-Supervised Representationscited by 81
  2. 2020Voice Conversion Challenge 2020: Intra-lingual semi-parallel and cross-lingual voice conversioncited by 75
  3. 2021Large-scale Self-Supervised Speech Representation Learning for Automatic Speaker Verificationcited by 71
  4. 2021VQMIVC: Vector Quantization and Mutual Information-Based Unsupervised Speech Representation Disentanglement for One-shot Voice Conversioncited by 63
  5. 2020AGAIN-VC: A One-shot Voice Conversion using Activation Guidance and Adaptive Instance Normalizationcited by 44
  6. 2020Any-to-Many Voice Conversion with Location-Relative Sequence-to-Sequence Modelingcited by 42
  7. 2020VQVC+: One-Shot Voice Conversion by Vector Quantization and U-Net architecturecited by 34
  8. 2022SpeechSplit 2.0: Unsupervised speech disentanglement for voice conversion Without tuning autoencoder Bottleneckscited by 27
  9. 2021S3PRL-VC: Open-source Voice Conversion Framework with Self-supervised Speech Representationscited by 22
  10. 2020Transfer Learning from Speech Synthesis to Voice Conversion with Non-Parallel Training Datacited by 17
  11. 2020Cotatron: Transcription-Guided Speech Encoder for Any-to-Many Voice Conversion without Parallel Datacited by 14
  12. 2018Collapsed speech segment detection and suppression for WaveNet vocodercited by 2
  13. 2022GlowVC: Mel-spectrogram space disentangling model for language-independent text-free voice conversioncited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 10; 2024: 24; 2025: 30; 2026: 15.

79 papers.

Sort:

By area:

  1. 2024MaskGCT: Zero-Shot Text-to-Speech with Masked Generative Codec Transformercited by 159
  2. 2023Voice Conversion With Just Nearest Neighborscited by 67
  3. 2024Zero-shot Voice Conversion with Diffusion Transformerscited by 39
  4. 2023OpenVoice: Versatile Instant Voice Cloningcited by 30
  5. 2024Coding Speech through Vocal Tract Kinematicscited by 18
  6. 2023Susceptibility to Influence of Large Language Modelscited by 17
  7. 2024PolyGlotFake: A Novel Multilingual and Multimodal DeepFake Datasetcited by 16
  8. 2024StreamVC: Real-Time Low-Latency Voice Conversioncited by 15
  9. 2025Predicting Turn-Taking and Backchannel in Human-Machine Conversations Using Linguistic, Acoustic, and Visual Signalscited by 10
  10. 2023Fake the Real: Backdoor Attack on Deep Speech Classification via Voice Conversioncited by 9
  11. 2023Phoneme Hallucinator: One-shot Voice Conversion via Set Expansioncited by 9
  12. 2024Voice Attribute Editing with Text Promptcited by 9
38 more
  1. 2024CLAD: Robust Audio Deepfake Detection Against Manipulation Attacks with Contrastive Learningcited by 9
  2. 2024Enhancing Partially Spoofed Audio Localization with Boundary-aware Attention Mechanismcited by 9
  3. 2025GenVC: Self-Supervised Zero-Shot Voice Conversioncited by 8
  4. 2026The Third VoicePrivacy Challenge: Preserving Emotional Expressiveness and Linguistic Content in Voice Anonymizationcited by 8
  5. 2024Enhancing the Stability of LLM-based Speech Generation Systems through Self-Supervised Representationscited by 7
  6. 2024VoiceShop: A Unified Speech-to-Speech Framework for Identity-Preserving Zero-Shot Voice Editingcited by 7
  7. 2024Fake Artificial Intelligence Generated Contents (FAIGC): A Survey of Theories, Detection Methods, and Opportunitiescited by 7
  8. 2024VQ-CTAP: Cross-Modal Fine-Grained Sequence Representation Learning for Speech Processingcited by 7
  9. 2024vec2wav 2.0: Advancing Voice Conversion via Discrete Token Vocoderscited by 6
  10. 2024SaMoye: Zero-shot Singing Voice Conversion Model Based on Feature Disentanglement and Enhancementcited by 5
  11. 2024Improving Pronunciation and Accent Conversion through Knowledge Distillation And Synthetic Ground-Truth from Native TTScited by 5
  12. 2025An Exhaustive Evaluation of TTS- and VC-based Data Augmentation for ASRcited by 5
  13. 2025SpecWav-Attack: Leveraging Spectrogram Resizing and Wav2Vec 2.0 for Attacking Anonymized Speechcited by 5
  14. 2023Voice Conversion for Stuttered Speech, Instruments, Unseen Languages and Textually Described Voicescited by 4
  15. 2023Vulnerability of Automatic Identity Recognition to Audio-Visual Deepfakescited by 4
  16. 2024SF-Speech: Straightened Flow for Zero-Shot Voice Clonecited by 4
  17. 2024Optimal Transport Maps are Good Voice Converterscited by 4
  18. 2025ReFlow-VC: Zero-shot Voice Conversion Based on Rectified Flow and Speaker Feature Optimizationcited by 4
  19. 2025RT-VC: Real-Time Zero-Shot Voice Conversion with Speech Articulatory Codingcited by 4
  20. 2026TVTSyn: Content-Synchronous Time-Varying Timbre for Streaming Voice Conversion and Anonymizationcited by 4
  21. 2024SoundMorpher: Perceptually-Uniform Sound Morphing with Diffusion Modelcited by 3
  22. 2025LinearVC: Linear transformations of self-supervised features through the lens of voice conversioncited by 3
  23. 2025DeepfakeBench-MM: A Comprehensive Benchmark for Multimodal Deepfake Detectioncited by 3
  24. 2024MulliVC: Multi-lingual Voice Conversion With Cycle Consistencycited by 2
  25. 2025Unispeaker: A Unified Approach for Multimodality-driven Speaker Generationcited by 2
  26. 2025Emotion Recognition and Generation: A Comprehensive Review of Face, Speech, and Text Modalitiescited by 2
  27. 2025Towards Better Disentanglement in Non-Autoregressive Zero-Shot Expressive Voice Conversioncited by 2
  28. 2025Training-Free Voice Conversion with Factorized Optimal Transportcited by 2
  29. 2025De-AntiFake: Rethinking the Protective Perturbations Against Voice Cloning Attackscited by 2
  30. 2025REF-VC: Robust, Expressive and Fast Zero-Shot Voice Conversion with Diffusion Transformerscited by 2
  31. 2025A Survey of Threats Against Voice Authentication and Anti-Spoofing Systemscited by 2
  32. 2025Mixture of Low-Rank Adapter Experts in Generalizable Audio Deepfake Detectioncited by 2
  33. 2025Objective Evaluation of Prosody and Intelligibility in Speech Synthesis via Conditional Prediction of Discrete Tokenscited by 2
  34. 2026The Voice of Equity: A Systematic Evaluation of Bias Mitigation Techniques for Speech-Based Cognitive Impairment Detection Across Architectures and Demographicscited by 2
  35. 2026X-VC: Zero-shot Streaming Voice Conversion in Codec Spacecited by 2
  36. 2023EventTrojan: Manipulating Non-Intrusive Speech Quality Assessment via Imperceptible Eventscited by 1
  37. 2024CrossVoice: Crosslingual Prosody Preserving Cascade-S2ST using Transfer Learningcited by 1
  38. 2024QR-VC: Leveraging Quantization Residuals for Linear Disentanglement in Zero-Shot Voice Conversioncited by 1

50 most cited of 79 All 79 (JSON)