mlsanityData to 7 Oct 2026

DiffSVC: A Diffusion Probabilistic Model for Singing Voice Conversion

arXiv 2105.13871 ↗First posted 28 May 2021eess.AS cs.CL cs.SDcited by 39

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

Singing voice conversion (SVC) is one promising technique which can enrich the way of human-computer interaction by endowing a computer the ability to produce high-fidelity and expressive singing voice. In this paper, we propose DiffSVC, an SVC system based on denoising diffusion probabilistic model. DiffSVC uses phonetic posteriorgrams (PPGs) as content features. A denoising module is trained in DiffSVC, which takes destroyed mel spectrogram produced by the diffusion/forward process and its corresponding step information as input to predict the added Gaussian noise. We use PPGs, fundamental frequency features and loudness features as auxiliary input to assist the denoising process. Experiments show that DiffSVC can achieve superior conversion performance in terms of naturalness and voice similarity to current state-of-the-art SVC approaches.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 20 of its 29 references that are in the map. First posted on arXiv 28 May 2021.

Builds on

20 of 29 references.

Closest ancestors · 1

Same direction, newest first

  1. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesis21 Sep

Other references · 19

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2020Denoising Diffusion Implicit Modelscited by 9,655
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  6. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  7. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  8. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  9. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 620
  10. 2018Efficient Neural Audio Synthesiscited by 218
  11. 2018Deep-FSMN for Large Vocabulary Continuous Speech Recognitioncited by 28
  12. 2020Unsupervised Cross-Domain Singing Voice Conversioncited by 23
7 more
  1. 2020FastSVC: Fast Cross-Domain Singing Voice Conversion with Feature-wise Linear Modulationcited by 21
  2. 2019Singing Voice Conversion with Disentangled Representations of Singer and Vocal Technique Using Variational Autoencoderscited by 18
  3. 2019PitchNet: Unsupervised Singing Voice Conversion with Pitch Adversarial Networkcited by 15
  4. 2019Unsupervised Singing Voice Conversioncited by 14
  5. 2020PPG-based singing voice conversion with adversarial representation learningcited by 12
  6. 2019Singing voice conversion with non-parallel datacited by 8
  7. 2020VAW-GAN for Singing Voice Conversion with Non-parallel Training Datacited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 7; 2023: 7; 2024: 9; 2025: 8; 2026: 6.

39 papers.

Sort:

By area:

  1. 2022A Survey on Generative Diffusion Modelcited by 238
  2. 2022Conditional Diffusion Probabilistic Model for Speech Enhancementcited by 132
  3. 2021Diffusion-Based Voice Conversion with Fast Maximum Likelihood Sampling Schemecited by 99
  4. 2023AUDIT: Audio Editing by Following Instructions with Latent Diffusion Modelscited by 81
  5. 2023A Survey on Audio Diffusion Models: Text To Speech Synthesis and Enhancement in Generative AIcited by 68
  6. 2021A Study on Speech Enhancement Based on Diffusion Probabilistic Modelcited by 52
  7. 2022Multi-instrument Music Synthesis with Spectrogram Diffusioncited by 42
  8. 2023DDDM-VC: Decoupled Denoising Diffusion Models with Disentangled Representation and Prior Mixup for Verified Robust Voice Conversioncited by 38
  9. 2023The Singing Voice Conversion Challenge 2023cited by 25
  10. 2023LightGrad: Lightweight Diffusion Probabilistic Model for Text-to-Speechcited by 13
  11. 2024CoMoSVC: Consistency Model-based Singing Voice Conversioncited by 11
  12. 2022Learning the Beauty in Songs: Neural Singing Voice Beautifiercited by 10
27 more
  1. 2022A Flexible Diffusion Modelcited by 10
  2. 2022Creative Painting with Latent Diffusion Modelscited by 8
  3. 2024FastVoiceGrad: One-step Diffusion-Based Voice Conversion with Adversarial Conditional Diffusion Distillationcited by 7
  4. 2024LDM-SVC: Latent Diffusion Model Based Zero-Shot Any-to-Any Singing Voice Conversion with Singer Guidancecited by 6
  5. 2022Robust One-Shot Singing Voice Conversioncited by 5
  6. 2024DIFFRENT: A Diffusion Model for Recording Environment Transfer of Speechcited by 3
  7. 2024SPA-SVC: Self-supervised Pitch Augmentation for Singing Voice Conversioncited by 3
  8. 2023A Comparative Study of Voice Conversion Models with Large-Scale Speech and Singing Data: The T13 Systems for the Singing Voice Conversion Challenge 2023cited by 2
  9. 2024SingVisio: Visual Analytics of Diffusion Model for Singing Voice Conversioncited by 2
  10. 2024LCM-SVC: Latent Diffusion Model Based Singing Voice Conversion with Inference Acceleration via Latent Consistency Distillationcited by 2
  11. 2025DAFMSVC: One-Shot Singing Voice Conversion with Dual Attention Mechanism and Flow Matchingcited by 2
  12. 2025LatentVoiceGrad: Nonparallel Voice Conversion with Latent Diffusion/Flow-Matching Modelscited by 2
  13. 2025HQ-SVC: Towards High-Quality Zero-Shot Singing Voice Conversion in Low-Resource Scenarioscited by 2
  14. 2024Converting Anyone's Voice: End-to-End Expressive Voice Conversion with a Conditional Diffusion Modelcited by 1
  15. 2025Singing Voice Conversion with Accompaniment Using Self-Supervised Representation-Based Melody Featurescited by 1
  16. 2025kNN-SVC: Robust Zero-Shot Singing Voice Conversion with Additive Synthesis and Concatenation Smoothness Optimizationcited by 1
  17. 2025VibE-SVC: Vibrato Extraction with High-frequency F0 Contour for Singing Voice Conversioncited by 1
  18. 2026MeanVoiceFlow: One-step Nonparallel Voice Conversion with Mean Flowscited by 1
  19. 2026VoiceDesigner: Text-to-Voice Generation and Editing via Unified Diffusion Modeling and Data Augmentationcited by 1
  20. 2023Effects of Convolutional Autoencoder Bottleneck Width on StarGAN-based Singing Technique Conversioncited by 0
  21. 2024CoDiff-VC: A Codec-Assisted Diffusion Model for Zero-shot Voice Conversioncited by 0
  22. 2025USM-VC: Mitigating Timbre Leakage with Universal Semantic Mapping Residual Block for Voice Conversioncited by 0
  23. 2025Degrading Voice: A Comprehensive Overview of Robust Voice Conversion Through Input Manipulationcited by 0
  24. 2026Controllable Singing Style Conversion with Boundary-Aware Information Bottleneckcited by 0
  25. 2026Towards Unified Song Generation and Singing Voice Conversion with Accompaniment Co-Generationcited by 0
  26. 2026Vibrato Expression Control for Singing Voice Conversion with Improving Independent Controlcited by 0
  27. 2026MeanVoiceFlow2: Joint Optimization of Mean Flow and Content Encoder for Fast One-Step Zero-Shot Voice Conversioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2021DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanism
  2. 2024RDSinger: Reference-based Diffusion Network for Singing Voice Synthesis
  3. 2024LHQ-SVC: Lightweight and High Quality Singing Voice Conversion Modeling
  4. 2025REF-VC: Robust, Expressive and Fast Zero-Shot Voice Conversion with Diffusion Transformers
  5. 2025SmoothSinger: A Conditional Diffusion Model for Singing Voice Synthesis with Multi-Resolution Architecture
  6. 2025DiTSinger: Scaling Singing Voice Synthesis with Diffusion Transformer and Implicit Alignment
  7. 2026Adapting Diffusion-Based Music Synthesis to Speech and Singing Voice Conversion
  8. 2023CoMoSpeech: One-Step Speech and Singing Voice Synthesis via Consistency Model