# Unsupervised Cross-Domain Singing Voice Conversion

arXiv: https://arxiv.org/abs/2008.02830 · HTML: https://mlsanity.com/paper/2008.02830/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Aug 2020.

## Built on by (most cited first, 23 of 23)

- [Generative Spoken Language Modeling from Raw Audio](https://mlsanity.com/paper/2102.01192/) (2021)
- [Speech Resynthesis from Discrete Disentangled Self-Supervised Representations](https://mlsanity.com/paper/2104.00355/) (2021)
- [Text-Free Prosody-Aware Generative Spoken Language Modeling](https://mlsanity.com/paper/2109.03264/) (2021)
- [AUDIT: Audio Editing by Following Instructions with Latent Diffusion Models](https://mlsanity.com/paper/2304.00830/) (2023)
- [StarGANv2-VC: A Diverse, Unsupervised, Non-parallel Framework for Natural-Sounding Voice Conversion](https://mlsanity.com/paper/2107.10394/) (2021)
- [DiffSVC: A Diffusion Probabilistic Model for Singing Voice Conversion](https://mlsanity.com/paper/2105.13871/) (2021)
- [Textless Speech Emotion Conversion using Discrete and Decomposed Representations](https://mlsanity.com/paper/2111.07402/) (2021)
- [The Singing Voice Conversion Challenge 2023](https://mlsanity.com/paper/2306.14422/) (2023)
- [fairseq S^2: A Scalable and Integrable Speech Synthesis Toolkit](https://mlsanity.com/paper/2109.06912/) (2021)
- [The Ethical Implications of Generative Audio Models: A Systematic Literature Review](https://mlsanity.com/paper/2307.05527/) (2023)
- [FSD: An Initial Chinese Dataset for Fake Song Detection](https://mlsanity.com/paper/2309.02232/) (2023)
- [CoMoSVC: Consistency Model-based Singing Voice Conversion](https://mlsanity.com/paper/2401.01792/) (2024)
- [Hierarchical disentangled representation learning for singing voice conversion](https://mlsanity.com/paper/2101.06842/) (2021)
- [Towards High-fidelity Singing Voice Conversion with Acoustic Reference and Contrastive Predictive Coding](https://mlsanity.com/paper/2110.04754/) (2021)
- [Robust One-Shot Singing Voice Conversion](https://mlsanity.com/paper/2210.11096/) (2022)
- [LDM-SVC: Latent Diffusion Model Based Zero-Shot Any-to-Any Singing Voice Conversion with Singer Guidance](https://mlsanity.com/paper/2406.05325/) (2024)
- [SaMoye: Zero-shot Singing Voice Conversion Model Based on Feature Disentanglement and Enhancement](https://mlsanity.com/paper/2407.07728/) (2024)
- [Rapping-Singing Voice Synthesis based on Phoneme-level Prosody Control](https://mlsanity.com/paper/2111.09146/) (2021)
- [Iteratively Improving Speech Recognition and Voice Conversion](https://mlsanity.com/paper/2305.15055/) (2023)
- [A Comparative Study of Voice Conversion Models with Large-Scale Speech and Singing Data: The T13 Systems for the Singing Voice Conversion Challenge 2023](https://mlsanity.com/paper/2310.05203/) (2023)
- [LCM-SVC: Latent Diffusion Model Based Singing Voice Conversion with Inference Acceleration via Latent Consistency Distillation](https://mlsanity.com/paper/2408.12354/) (2024)
- [A Comparative Analysis Of Latent Regressor Losses For Singing Voice Conversion](https://mlsanity.com/paper/2302.13678/) (2023)
- [Are Music Foundation Models Better at Singing Voice Deepfake Detection? Far-Better Fuse them with Speech Foundation Models](https://mlsanity.com/paper/2409.14131/) (2024)

## Abstract (arXiv)

We present a wav-to-wav generative model for the task of singing voice conversion from any identity. Our method utilizes both an acoustic model, trained for the task of automatic speech recognition, together with melody extracted features to drive a waveform-based generator. The proposed generative architecture is invariant to the speaker's identity and can be trained to generate target singers from unlabeled training data, using either speech or singing sources. The model is optimized in an end-to-end fashion without any manual supervision, such as lyrics, musical notes or parallel samples. The proposed approach is fully-convolutional and can generate audio in real-time. Experiments show that our method significantly outperforms the baseline methods while generating convincingly better audio samples than alternative attempts.