mlsanityData to 7 Oct 2026

FastVoiceGrad: One-step Diffusion-Based Voice Conversion with Adversarial Conditional Diffusion Distillation

arXiv 2409.02245 ↗First posted 3 Sep 2024cs.SD cs.AI cs.LGcited by 7

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

Diffusion-based voice conversion (VC) techniques such as VoiceGrad have attracted interest because of their high VC performance in terms of speech quality and speaker similarity. However, a notable limitation is the slow inference caused by the multi-step reverse diffusion. Therefore, we propose FastVoiceGrad, a novel one-step diffusion-based VC that reduces the number of iterations from dozens to one while inheriting the high VC performance of the multi-step diffusion-based VC. We obtain the model using adversarial conditional diffusion distillation (ACDD), leveraging the ability of generative adversarial networks and diffusion models while reconsidering the initial states in sampling. Evaluations of one-shot any-to-any VC demonstrate that FastVoiceGrad achieves VC performance superior to or comparable to that of previous multi-step diffusion-based VC while enhancing the inference speed. Audio samples are available at https://www.kecl.ntt.co.jp/people/kaneko.takuhiro/projects/fastvoicegrad/.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 38 of its 46 references that are in the map. First posted on arXiv 3 Sep 2024.

Builds on

38 of 46 references.

Closest ancestors · 3

Same direction, newest first

  1. 2021Diffusion-Based Voice Conversion with Fast Maximum Likelihood Sampling Scheme28 Sep
  2. 2021DiffSVC: A Diffusion Probabilistic Model for Singing Voice Conversion28 May
  3. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesis21 Sep

Other references · 35

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  6. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  7. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  8. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  9. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  10. 2014NICE: Non-linear Independent Components Estimationcited by 1,052
  11. 2016Least Squares Generative Adversarial Networkscited by 1,000
  12. 2023Adversarial Diffusion Distillationcited by 741
23 more
  1. 2016Language Modeling with Gated Convolutional Networkscited by 688
  2. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 639
  3. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 620
  4. 2020ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verificationcited by 606
  5. 2016Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networkscited by 541
  6. 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 381
  7. 2019MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesiscited by 351
  8. 2018Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesiscited by 280
  9. 2020DNSMOS: A Non-Intrusive Perceptual Objective Speech Quality metric to evaluate Noise Suppressorscited by 235
  10. 2019One-shot Voice Conversion by Separating Speaker and Content Representations with Instance Normalizationcited by 82
  11. 2018StarGAN-VC: Non-parallel many-to-many voice conversion with star generative adversarial networkscited by 78
  12. 2021UnivNet: A Neural Vocoder with Multi-Resolution Spectrogram Discriminators for High-Fidelity Waveform Generationcited by 71
  13. 2021VQMIVC: Vector Quantization and Mutual Information-Based Unsupervised Speech Representation Disentanglement for One-shot Voice Conversioncited by 63
  14. 2021StarGANv2-VC: A Diverse, Unsupervised, Non-parallel Framework for Natural-Sounding Voice Conversioncited by 57
  15. 2022The VoiceMOS Challenge 2022cited by 57
  16. 2016Voice Conversion from Non-parallel Corpora Using Variational Auto-encodercited by 45
  17. 2020AGAIN-VC: A One-shot Voice Conversion using Activation Guidance and Adaptive Instance Normalizationcited by 44
  18. 2020Any-to-Many Voice Conversion with Location-Relative Sequence-to-Sequence Modelingcited by 42
  19. 2019StarGAN-VC2: Rethinking Conditional Methods for StarGAN-Based Voice Conversioncited by 37
  20. 2019Blow: a single-scale hyperconditioned flow for non-parallel raw-audio voice conversioncited by 29
  21. 2021MaskCycleGAN-VC: Learning Non-parallel Voice Conversion with Filling in Framescited by 18
  22. 2019Non-Parallel Voice Conversion with Cyclic Variational Autoencodercited by 15
  23. 2020VoiceGrad: Non-Parallel Any-to-Many Voice Conversion with Annealed Langevin Dynamicscited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 5; 2026: 2.

7 papers.

  1. 2025FasterVoiceGrad: Faster One-step Diffusion-Based Voice Conversion with Adversarial Diffusion Conversion Distillationcited by 2
  2. 2025Vocoder-Projected Feature Discriminatorcited by 2
  3. 2025LatentVoiceGrad: Nonparallel Voice Conversion with Latent Diffusion/Flow-Matching Modelscited by 2
  4. 2025Training-Free Multi-Step Audio Source Separationcited by 1
  5. 2026MeanVoiceFlow: One-step Nonparallel Voice Conversion with Mean Flowscited by 1
  6. 2025Combining Deterministic Enhanced Conditions with Dual-Streaming Encoding for Diffusion-Based Speech Enhancementcited by 0
  7. 2026MeanVoiceFlow2: Joint Optimization of Mean Flow and Content Encoder for Fast One-Step Zero-Shot Voice Conversioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Voice Conversion with Denoising Diffusion Probabilistic GAN Models
  2. 2023Diff-HierVC: Diffusion-based Hierarchical Voice Conversion with Robust Pitch Generation and Masked Prior for Zero-shot Speaker Adaptation
  3. 2023DiffVoice: Text-to-Speech with Latent Diffusion
  4. 2023DDDM-VC: Decoupled Denoising Diffusion Models with Disentangled Representation and Prior Mixup for Verified Robust Voice Conversion
  5. 2023A Survey on Audio Diffusion Models: Text To Speech Synthesis and Enhancement in Generative AI
  6. 2023Adversarial Training of Denoising Diffusion Model Using Dual Discriminators for High-Fidelity Multi-Speaker TTS
  7. 2022DiffGAN-TTS: High-Fidelity and Efficient Text-to-Speech with Denoising Diffusion GANs
  8. 2022FastDiff: A Fast Conditional Diffusion Model for High-Quality Speech Synthesis