mlsanityData to 7 Oct 2026

NVC-Net: End-to-End Adversarial Voice Conversion

arXiv 2106.00992 ↗First posted 2 Jun 2021cs.SD cs.AI eess.AScited by 9

Not on the map. Why?

Abstract (arXiv)

Voice conversion has gained increasing popularity in many applications of speech synthesis. The idea is to change the voice identity from one speaker into another while keeping the linguistic content unchanged. Many voice conversion approaches rely on the use of a vocoder to reconstruct the speech from acoustic features, and as a consequence, the speech quality heavily depends on such a vocoder. In this paper, we propose NVC-Net, an end-to-end adversarial network, which performs voice conversion directly on the raw audio waveform of arbitrary length. By disentangling the speaker identity from the speech content, NVC-Net is able to perform non-parallel traditional many-to-many voice conversion as well as zero-shot voice conversion from a short utterance of an unseen target speaker. Importantly, NVC-Net is non-autoregressive and fully convolutional, achieving fast inference. Our model is capable of producing samples at a rate of more than 3600 kHz on an NVIDIA V100 GPU, being orders of magnitude faster than state-of-the-art methods under the same hardware configurations. Objective and subjective evaluations on non-parallel many-to-many voice conversion tasks show that NVC-Net obtains competitive results with significantly fewer parameters.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 37 of its 53 references that are in the map. First posted on arXiv 2 Jun 2021.

Builds on

37 of 53 references.

References · 37

  1. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  2. 2013Auto-Encoding Variational Bayescited by 3,980
  3. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  4. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  5. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  6. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
  7. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,242
  8. 2019FaceForensics++: Learning to Detect Manipulated Facial Imagescited by 1,095
  9. 2017StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translationcited by 1,036
  10. 2016Least Squares Generative Adversarial Networkscited by 1,000
  11. 2016Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networkscited by 541
  12. 2015Autoencoding beyond pixels using a learned similarity metriccited by 407
25 more
  1. 2019MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesiscited by 351
  2. 2016A Learned Representation For Artistic Stylecited by 340
  3. 2019Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogramcited by 337
  4. 2018WaveGlow: A Flow-based Generative Network for Speech Synthesiscited by 308
  5. 2017Neural Audio Synthesis of Musical Notes with WaveNet Autoencoderscited by 222
  6. 2018Adversarial Audio Synthesiscited by 222
  7. 2019Few-Shot Unsupervised Image-to-Image Translationcited by 222
  8. 2018Efficient Neural Audio Synthesiscited by 218
  9. 2019AUTOVC: Zero-Shot Voice Style Transfer with Only Autoencoder Losscited by 189
  10. 2020DDSP: Differentiable Digital Signal Processingcited by 177
  11. 2020An Overview of Voice Conversion and its Challenges: From Statistical Modeling to Deep Learningcited by 111
  12. 2019High Fidelity Speech Synthesis with Adversarial Networkscited by 90
  13. 2019One-shot Voice Conversion by Separating Speaker and Content Representations with Instance Normalizationcited by 82
  14. 2018StarGAN-VC: Non-parallel many-to-many voice conversion with star generative adversarial networkscited by 78
  15. 2018The Voice Conversion Challenge 2018: Promoting Development of Parallel and Nonparallel Methodscited by 68
  16. 2016Voice Conversion from Non-parallel Corpora Using Variational Auto-encodercited by 45
  17. 2020DeepFake Detection: Current Challenges and Next Stepscited by 44
  18. 2017Voice Conversion from Unaligned Corpora using Variational Autoencoding Wasserstein Generative Adversarial Networkscited by 38
  19. 2019StarGAN-VC2: Rethinking Conditional Methods for StarGAN-Based Voice Conversioncited by 37
  20. 2017Parallel-Data-Free Voice Conversion Using Cycle-Consistent Adversarial Networkscited by 30
  21. 2019Blow: a single-scale hyperconditioned flow for non-parallel raw-audio voice conversioncited by 29
  22. 2020DeepSonar: Towards Effective and Robust Detection of AI-Synthesized Fake Voicescited by 24
  23. 2018High-quality nonparallel voice conversion based on cycle-consistent adversarial networkcited by 13
  24. 2020Voice Conversion by Cascading Automatic Speech Recognition and Text-to-Speech Synthesis with Prosody Transfercited by 10
  25. 2018Collapsed speech segment detection and suppression for WaveNet vocodercited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 2; 2023: 3; 2024: 1; 2025: 1; 2026: 1.

9 papers.

  1. 2021Training Robust Zero-Shot Voice Conversion Models with Self-supervised Featurescited by 7
  2. 2022Disentangling speech from surroundings with neural embeddingscited by 6
  3. 2023Towards General-Purpose Text-Instruction-Guided Voice Conversioncited by 6
  4. 2022HiFi-VC: High Quality ASR-Based Voice Conversioncited by 3
  5. 2023SLMGAN: Exploiting Speech Language Model Representations for Unsupervised Zero-Shot Voice Conversion in GANscited by 3
  6. 2023Reimagining Speech: A Scoping Review of Deep Learning-Powered Voice Conversioncited by 3
  7. 2025Fast-VGAN: Lightweight Voice Conversion with Explicit Control of F0 and Duration Parameterscited by 2
  8. 2024Converting Anyone's Voice: End-to-End Expressive Voice Conversion with a Conditional Diffusion Modelcited by 1
  9. 2026A survey of AI-generated voices and their detectioncited by 0