FasterVoiceGrad: Faster One-step Diffusion-Based Voice Conversion with Adversarial Diffusion Conversion Distillation
On the map: Diffusion models → Audio, speech and music
Abstract (arXiv)
A diffusion-based voice conversion (VC) model (e.g., VoiceGrad) can achieve high speech quality and speaker similarity; however, its conversion process is slow owing to iterative sampling. FastVoiceGrad overcomes this limitation by distilling VoiceGrad into a one-step diffusion model. However, it still requires a computationally intensive content encoder to disentangle the speaker's identity and content, which slows conversion. Therefore, we propose FasterVoiceGrad, a novel one-step diffusion-based VC model obtained by simultaneously distilling a diffusion model and content encoder using adversarial diffusion conversion distillation (ADCD), where distillation is performed in the conversion process while leveraging adversarial and score distillation training. Experimental evaluations of one-shot VC demonstrated that FasterVoiceGrad achieves competitive VC performance compared to FastVoiceGrad, with 6.6-6.9 and 1.8 times faster speed on a GPU and CPU, respectively.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 37 of its 44 references that are in the map. First posted on arXiv 25 Aug 2025.
Builds on
37 of 44 references.
Closest ancestors · 4
Same direction, newest first
- 2024FastVoiceGrad: One-step Diffusion-Based Voice Conversion with Adversarial Conditional Diffusion Distillation3 Sep
- 2023Diff-HierVC: Diffusion-based Hierarchical Voice Conversion with Robust Pitch Generation and Masked Prior for Zero-shot Speaker Adaptation8 Nov
- 2023DDDM-VC: Decoupled Denoising Diffusion Models with Disentangled Representation and Prior Mixup for Verified Robust Voice Conversion25 May
- 2021Diffusion-Based Voice Conversion with Fast Maximum Likelihood Sampling Scheme28 Sep
Other references · 33
- 2014Adam: A Method for Stochastic Optimizationcited by 37,240
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
- 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
- 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
- 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
- 2016Instance Normalization: The Missing Ingredient for Fast Stylizationcited by 1,198
- 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,154
- 2014NICE: Non-linear Independent Components Estimationcited by 1,058
- 2016Least Squares Generative Adversarial Networkscited by 1,001
- 2023Adversarial Diffusion Distillationcited by 759
21 more
- 2016Language Modeling with Gated Convolutional Networkscited by 702
- 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 649
- 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 601
- 2016Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networkscited by 543
- 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 390
- 2019MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesiscited by 354
- 2018Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesiscited by 282
- 2020DNSMOS: A Non-Intrusive Perceptual Objective Speech Quality metric to evaluate Noise Suppressorscited by 241
- 2019One-shot Voice Conversion by Separating Speaker and Content Representations with Instance Normalizationcited by 82
- 2018StarGAN-VC: Non-parallel many-to-many voice conversion with star generative adversarial networkscited by 78
- 2021UnivNet: A Neural Vocoder with Multi-Resolution Spectrogram Discriminators for High-Fidelity Waveform Generationcited by 74
- 2021VQMIVC: Vector Quantization and Mutual Information-Based Unsupervised Speech Representation Disentanglement for One-shot Voice Conversioncited by 64
- 2021StarGANv2-VC: A Diverse, Unsupervised, Non-parallel Framework for Natural-Sounding Voice Conversioncited by 58
- 2016Voice Conversion from Non-parallel Corpora Using Variational Auto-encodercited by 45
- 2020AGAIN-VC: A One-shot Voice Conversion using Activation Guidance and Adaptive Instance Normalizationcited by 44
- 2020Any-to-Many Voice Conversion with Location-Relative Sequence-to-Sequence Modelingcited by 42
- 2019StarGAN-VC2: Rethinking Conditional Methods for StarGAN-Based Voice Conversioncited by 37
- 2019Blow: a single-scale hyperconditioned flow for non-parallel raw-audio voice conversioncited by 29
- 2021MaskCycleGAN-VC: Learning Non-parallel Voice Conversion with Filling in Framescited by 18
- 2020VoiceGrad: Non-Parallel Any-to-Many Voice Conversion with Annealed Langevin Dynamicscited by 14
- 2024DreamVoice: Text-Guided Voice Conversioncited by 7
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025LatentVoiceGrad: Nonparallel Voice Conversion with Latent Diffusion/Flow-Matching Models
- 2023Voice Conversion with Denoising Diffusion Probabilistic GAN Models
- 2023DiffVoice: Text-to-Speech with Latent Diffusion
- 2024Zero-shot Voice Conversion with Diffusion Transformers
- 2025MeanVC: Lightweight and Streaming Zero-Shot Voice Conversion via Mean Flows
- 2023Adversarial Training of Denoising Diffusion Model Using Dual Discriminators for High-Fidelity Multi-Speaker TTS
- 2025ZSVC: Zero-shot Style Voice Conversion with Disentangled Latent Diffusion Models and Adversarial Training
- 2025ReFlow-VC: Zero-shot Voice Conversion Based on Rectified Flow and Speaker Feature Optimization