mlsanityData to 7 Oct 2026

NoiseVC: Towards High Quality Zero-Shot Voice Conversion

arXiv 2104.06074 ↗First posted 13 Apr 2021cs.SD cs.LG eess.AScited by 4

Not on the map. Why?

Abstract (arXiv)

Voice conversion (VC) is a task that transforms voice from target audio to source without losing linguistic contents, it is challenging especially when source and target speakers are unseen during training (zero-shot VC). Previous approaches require a pre-trained model or linguistic data to do the zero-shot conversion. Meanwhile, VC models with Vector Quantization (VQ) or Instance Normalization (IN) are able to disentangle contents from audios and achieve successful conversions. However, disentanglement in these models highly relies on heavily constrained bottleneck layers, thus, the sound quality is drastically sacrificed. In this paper, we propose NoiseVC, an approach that can disentangle contents based on VQ and Contrastive Predictive Coding (CPC). Additionally, Noise Augmentation is performed to further enhance disentanglement capability. We conduct several experiments and demonstrate that NoiseVC has a strong disentanglement ability with a small sacrifice of quality.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 18 of its 32 references that are in the map. First posted on arXiv 13 Apr 2021.

Builds on

18 of 32 references.

References · 18

  1. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  2. 2017Neural Discrete Representation Learningcited by 4,650
  3. 2019Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogramcited by 337
  4. 2019vq-wav2vec: Self-Supervised Learning of Discrete Speech Representationscited by 335
  5. 2017Improved Texture Networks: Maximizing Quality and Diversity in Feed-forward Stylization and Texture Synthesiscited by 215
  6. 2019AUTOVC: Zero-Shot Voice Style Transfer with Only Autoencoder Losscited by 189
  7. 2019Unsupervised speech representation learning using WaveNet autoencoderscited by 90
  8. 2019One-shot Voice Conversion by Separating Speaker and Content Representations with Instance Normalizationcited by 82
  9. 2018StarGAN-VC: Non-parallel many-to-many voice conversion with star generative adversarial networkscited by 78
  10. 2020Data Augmenting Contrastive Learning of Speech Representations in the Time Domaincited by 63
  11. 2020Vector-quantized neural networks for acoustic unit discovery in the ZeroSpeech 2020 challengecited by 55
  12. 2020AGAIN-VC: A One-shot Voice Conversion using Activation Guidance and Adaptive Instance Normalizationcited by 44
6 more
  1. 2017Voice Conversion from Unaligned Corpora using Variational Autoencoding Wasserstein Generative Adversarial Networkscited by 38
  2. 2020VQVC+: One-Shot Voice Conversion by Vector Quantization and U-Net architecturecited by 34
  3. 2017Parallel-Data-Free Voice Conversion Using Cycle-Consistent Adversarial Networkscited by 30
  4. 2019Blow: a single-scale hyperconditioned flow for non-parallel raw-audio voice conversioncited by 29
  5. 2020Transfer Learning from Speech Synthesis to Voice Conversion with Non-Parallel Training Datacited by 17
  6. 2019Joint training framework for text-to-speech and voice conversion using multi-source Tacotron and WaveNetcited by 15

Built on by

Cited by year

Citing papers per year: 2021: 3; 2022: 0; 2023: 1.

4 papers.

  1. 2021YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyonecited by 277
  2. 2023SEF-VC: Speaker Embedding Free Zero-Shot Voice Conversion with Cross Attentioncited by 19
  3. 2021Multi-view Contrastive Self-Supervised Learning of Accounting Data Representations for Downstream Audit Taskscited by 8
  4. 2021Zero-shot Voice Conversion via Self-supervised Prosody Representation Learningcited by 6