# DiffSVC: A Diffusion Probabilistic Model for Singing Voice Conversion

arXiv: https://arxiv.org/abs/2105.13871 · HTML: https://mlsanity.com/paper/2105.13871/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Placed in Audio, speech and music, one of 14 directions in Diffusion models.
- Builds on 20 of its 29 references that are in the map.
- First posted on arXiv 28 May 2021.

## Placed in

- Diffusion models / [Audio, speech and music](https://mlsanity.com/diffusion/audio-speech-and-music/)

## Builds on (most cited first, 20 of 20)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [DiffWave: A Versatile Diffusion Model for Audio Synthesis](https://mlsanity.com/paper/2009.09761/) (2020)
- [HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis](https://mlsanity.com/paper/2010.05646/) (2020)
- [WaveGrad: Estimating Gradients for Waveform Generation](https://mlsanity.com/paper/2009.00713/) (2020)
- [Efficient Neural Audio Synthesis](https://mlsanity.com/paper/1802.08435/) (2018)
- [Deep-FSMN for Large Vocabulary Continuous Speech Recognition](https://mlsanity.com/paper/1803.05030/) (2018)
- [Unsupervised Cross-Domain Singing Voice Conversion](https://mlsanity.com/paper/2008.02830/) (2020)
- [FastSVC: Fast Cross-Domain Singing Voice Conversion with Feature-wise Linear Modulation](https://mlsanity.com/paper/2011.05731/) (2020)
- [Singing Voice Conversion with Disentangled Representations of Singer and Vocal Technique Using Variational Autoencoders](https://mlsanity.com/paper/1912.02613/) (2019)
- [PitchNet: Unsupervised Singing Voice Conversion with Pitch Adversarial Network](https://mlsanity.com/paper/1912.01852/) (2019)
- [Unsupervised Singing Voice Conversion](https://mlsanity.com/paper/1904.06590/) (2019)
- [PPG-based singing voice conversion with adversarial representation learning](https://mlsanity.com/paper/2010.14804/) (2020)
- [Singing voice conversion with non-parallel data](https://mlsanity.com/paper/1903.04124/) (2019)
- [VAW-GAN for Singing Voice Conversion with Non-parallel Training Data](https://mlsanity.com/paper/2008.03992/) (2020)

## Built on by (most cited first, 30 of 39)

- [A Survey on Generative Diffusion Model](https://mlsanity.com/paper/2209.02646/) (2022)
- [Conditional Diffusion Probabilistic Model for Speech Enhancement](https://mlsanity.com/paper/2202.05256/) (2022)
- [Diffusion-Based Voice Conversion with Fast Maximum Likelihood Sampling Scheme](https://mlsanity.com/paper/2109.13821/) (2021)
- [AUDIT: Audio Editing by Following Instructions with Latent Diffusion Models](https://mlsanity.com/paper/2304.00830/) (2023)
- [A Survey on Audio Diffusion Models: Text To Speech Synthesis and Enhancement in Generative AI](https://mlsanity.com/paper/2303.13336/) (2023)
- [A Study on Speech Enhancement Based on Diffusion Probabilistic Model](https://mlsanity.com/paper/2107.11876/) (2021)
- [Multi-instrument Music Synthesis with Spectrogram Diffusion](https://mlsanity.com/paper/2206.05408/) (2022)
- [DDDM-VC: Decoupled Denoising Diffusion Models with Disentangled Representation and Prior Mixup for Verified Robust Voice Conversion](https://mlsanity.com/paper/2305.15816/) (2023)
- [The Singing Voice Conversion Challenge 2023](https://mlsanity.com/paper/2306.14422/) (2023)
- [LightGrad: Lightweight Diffusion Probabilistic Model for Text-to-Speech](https://mlsanity.com/paper/2308.16569/) (2023)
- [CoMoSVC: Consistency Model-based Singing Voice Conversion](https://mlsanity.com/paper/2401.01792/) (2024)
- [Learning the Beauty in Songs: Neural Singing Voice Beautifier](https://mlsanity.com/paper/2202.13277/) (2022)
- [A Flexible Diffusion Model](https://mlsanity.com/paper/2206.10365/) (2022)
- [Creative Painting with Latent Diffusion Models](https://mlsanity.com/paper/2209.14697/) (2022)
- [FastVoiceGrad: One-step Diffusion-Based Voice Conversion with Adversarial Conditional Diffusion Distillation](https://mlsanity.com/paper/2409.02245/) (2024)
- [Robust One-Shot Singing Voice Conversion](https://mlsanity.com/paper/2210.11096/) (2022)
- [LDM-SVC: Latent Diffusion Model Based Zero-Shot Any-to-Any Singing Voice Conversion with Singer Guidance](https://mlsanity.com/paper/2406.05325/) (2024)
- [DIFFRENT: A Diffusion Model for Recording Environment Transfer of Speech](https://mlsanity.com/paper/2401.08102/) (2024)
- [SPA-SVC: Self-supervised Pitch Augmentation for Singing Voice Conversion](https://mlsanity.com/paper/2406.05692/) (2024)
- [A Comparative Study of Voice Conversion Models with Large-Scale Speech and Singing Data: The T13 Systems for the Singing Voice Conversion Challenge 2023](https://mlsanity.com/paper/2310.05203/) (2023)
- [SingVisio: Visual Analytics of Diffusion Model for Singing Voice Conversion](https://mlsanity.com/paper/2402.12660/) (2024)
- [LCM-SVC: Latent Diffusion Model Based Singing Voice Conversion with Inference Acceleration via Latent Consistency Distillation](https://mlsanity.com/paper/2408.12354/) (2024)
- [DAFMSVC: One-Shot Singing Voice Conversion with Dual Attention Mechanism and Flow Matching](https://mlsanity.com/paper/2508.05978/) (2025)
- [LatentVoiceGrad: Nonparallel Voice Conversion with Latent Diffusion/Flow-Matching Models](https://mlsanity.com/paper/2509.08379/) (2025)
- [HQ-SVC: Towards High-Quality Zero-Shot Singing Voice Conversion in Low-Resource Scenarios](https://mlsanity.com/paper/2511.08496/) (2025)
- [Converting Anyone's Voice: End-to-End Expressive Voice Conversion with a Conditional Diffusion Model](https://mlsanity.com/paper/2405.01730/) (2024)
- [Singing Voice Conversion with Accompaniment Using Self-Supervised Representation-Based Melody Features](https://mlsanity.com/paper/2502.04722/) (2025)
- [kNN-SVC: Robust Zero-Shot Singing Voice Conversion with Additive Synthesis and Concatenation Smoothness Optimization](https://mlsanity.com/paper/2504.05686/) (2025)
- [VibE-SVC: Vibrato Extraction with High-frequency F0 Contour for Singing Voice Conversion](https://mlsanity.com/paper/2505.20794/) (2025)
- [MeanVoiceFlow: One-step Nonparallel Voice Conversion with Mean Flows](https://mlsanity.com/paper/2602.18104/) (2026)

## Abstract (arXiv)

Singing voice conversion (SVC) is one promising technique which can enrich the way of human-computer interaction by endowing a computer the ability to produce high-fidelity and expressive singing voice. In this paper, we propose DiffSVC, an SVC system based on denoising diffusion probabilistic model. DiffSVC uses phonetic posteriorgrams (PPGs) as content features. A denoising module is trained in DiffSVC, which takes destroyed mel spectrogram produced by the diffusion/forward process and its corresponding step information as input to predict the added Gaussian noise. We use PPGs, fundamental frequency features and loudness features as auxiliary input to assist the denoising process. Experiments show that DiffSVC can achieve superior conversion performance in terms of naturalness and voice similarity to current state-of-the-art SVC approaches.