mlsanityData to 14 Oct 2026

AUTOVC: Zero-Shot Voice Style Transfer with Only Autoencoder Loss

arXiv 1905.05879 ↗First posted 14 May 2019eess.AS cs.AI cs.LGcited by 192

Not on the map. Why?

Abstract (arXiv)

Non-parallel many-to-many voice conversion, as well as zero-shot voice conversion, remain under-explored areas. Deep style transfer algorithms, such as generative adversarial networks (GAN) and conditional variational autoencoder (CVAE), are being applied as new solutions in this field. However, GAN training is sophisticated and difficult, and there is no strong evidence that its generated speech is of good perceptual quality. On the other hand, CVAE training is simple but does not come with the distribution-matching property of a GAN. In this paper, we propose a new style transfer scheme that involves only an autoencoder with a carefully designed bottleneck. We formally show that this scheme can achieve distribution-matching style transfer by training only on a self-reconstruction loss. Based on this scheme, we proposed AUTOVC, which achieves state-of-the-art results in many-to-many voice conversion with non-parallel data, and which is the first to perform zero-shot voice conversion.

Key facts. Cited by 192 papers in the map (data through 14 Oct 2026). First posted on arXiv 14 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2019: 1; 2020: 0; 2021: 37; 2022: 35; 2023: 39; 2024: 36; 2025: 31; 2026: 13.

192 papers.

Sort:

By area:

  1. 2021YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyonecited by 278
  2. 2024NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Modelscited by 226
  3. 2024MaskGCT: Zero-Shot Text-to-Speech with Masked Generative Codec Transformercited by 161
  4. 2021Diffusion-Based Voice Conversion with Fast Maximum Likelihood Sampling Schemecited by 100
  5. 2023SpeechTokenizer: Unified Speech Tokenizer for Speech Large Language Modelscited by 87
  6. 2021Deepfakes Generation and Detection: State-of-the-art, open challenges, countermeasures, and way forwardcited by 82
  7. 2021Neural Analysis and Synthesis: Reconstructing Speech from Self-Supervised Representationscited by 81
  8. 2022DFA-NeRF: Personalized Talking Head Generation via Disentangled Face Attributes Neural Renderingcited by 80
  9. 2022FreeVC: Towards High-Quality Text-Free One-Shot Voice Conversioncited by 80
  10. 2022ContentVec: An Improved Self-Supervised Speech Representation by Disentangling Speakerscited by 74
  11. 2023Voice Conversion With Just Nearest Neighborscited by 68
  12. 2021VQMIVC: Vector Quantization and Mutual Information-Based Unsupervised Speech Representation Disentanglement for One-shot Voice Conversioncited by 64
38 more
  1. 2021StarGANv2-VC: A Diverse, Unsupervised, Non-parallel Framework for Natural-Sounding Voice Conversioncited by 58
  2. 2023Mega-TTS: Zero-Shot Text-to-Speech at Scale with Intrinsic Inductive Biascited by 51
  3. 2023PromptTTS 2: Describing and Generating Voices with Text Promptcited by 48
  4. 2024Robustifying Safety-Aligned Large Language Models through Clean Data Curationcited by 44
  5. 2024Zero-shot Voice Conversion with Diffusion Transformerscited by 43
  6. 2025Recent Advances in Discrete Speech Tokens: A Reviewcited by 43
  7. 2024A Large-Scale Evaluation of Speech Foundation Modelscited by 40
  8. 2023DDDM-VC: Decoupled Denoising Diffusion Models with Disentangled Representation and Prior Mixup for Verified Robust Voice Conversioncited by 38
  9. 2024Foundation Models for Music: A Surveycited by 37
  10. 2021Live Speech Portraits: Real-Time Photorealistic Talking-Head Animationcited by 36
  11. 2023HierSpeech++: Bridging the Gap between Semantic and Acoustic Representation of Speech by Hierarchical Variational Inference for Zero-shot Speech Synthesiscited by 35
  12. 2022iSTFTNet: Fast and Lightweight Mel-Spectrogram Vocoder Incorporating Inverse Short-Time Fourier Transformcited by 32
  13. 2023Diff-HierVC: Diffusion-based Hierarchical Voice Conversion with Robust Pitch Generation and Masked Prior for Zero-shot Speaker Adaptationcited by 30
  14. 2022SpeechSplit 2.0: Unsupervised speech disentanglement for voice conversion Without tuning autoencoder Bottleneckscited by 28
  15. 2022Robust Disentangled Variational Speech Representation Learning for Zero-shot Voice Conversioncited by 28
  16. 2022Audio-Visual Speech Codecs: Rethinking Audio-Visual Speech Enhancement by Re-Synthesiscited by 24
  17. 2024SafeEar: Content Privacy-Preserving Audio Deepfake Detectioncited by 21
  18. 2021Multimodal Emotion Recognition with High-level Speech and Text Featurescited by 20
  19. 2022Zero-Shot Voice Conditioning for Denoising Diffusion TTS Modelscited by 20
  20. 2021MaskCycleGAN-VC: Learning Non-parallel Voice Conversion with Filling in Framescited by 18
  21. 2021STYLER: Style Factor Modeling with Rapidity and Robustness via Speech Decomposition for Expressive and Controllable Neural Text to Speechcited by 18
  22. 2022Face-Dubbing++: Lip-Synchronous, Voice Preserving Translation of Videoscited by 18
  23. 2021SpeechNet: A Universal Modularized Model for Speech Processing Taskscited by 16
  24. 2022Speaking Style Conversion in the Waveform Domain Using Discrete Self-Supervised Unitscited by 15
  25. 2023ACE-VC: Adaptive and Controllable Voice Conversion using Explicitly Disentangled Self-supervised Speech Representationscited by 15
  26. 2024StreamVC: Real-Time Low-Latency Voice Conversioncited by 15
  27. 2021Adversarially learning disentangled speech representations for robust multi-factor voice conversioncited by 14
  28. 2021Voice Conversion Can Improve ASR in Very Low-Resource Settingscited by 14
  29. 2022Speech Representation Disentanglement with Adversarial Mutual Information Learning for One-shot Voice Conversioncited by 14
  30. 2024StableVC: Style Controllable Zero-Shot Voice Conversion with Conditional Flow Matchingcited by 14
  31. 2024Text-to-Song: Towards Controllable Music Generation Incorporating Vocals and Accompanimentcited by 13
  32. 2021"Hello, It's Me": Deep Learning-based Speech Synthesis Attacks in the Real Worldcited by 12
  33. 2021Assem-VC: Realistic Voice Conversion by Assembling Modern Speech Synthesis Techniquescited by 11
  34. 2021Style Equalization: Unsupervised Learning of Controllable Generative Sequence Modelscited by 11
  35. 2022VarietySound: Timbre-Controllable Video to Sound Generation via Unsupervised Information Disentanglementcited by 11
  36. 2023HiFTNet: A Fast High-Quality Neural Vocoder with Harmonic-plus-Noise Filter and Inverse Short Time Fourier Transformcited by 11
  37. 2024FreGrad: Lightweight and Fast Frequency-aware Diffusion Vocodercited by 11
  38. 2022Learning the Beauty in Songs: Neural Singing Voice Beautifiercited by 10

50 most cited of 192 All 192 (JSON)