# Singing Voice Conversion with Disentangled Representations of Singer and Vocal Technique Using Variational Autoencoders

arXiv: https://arxiv.org/abs/1912.02613 · HTML: https://mlsanity.com/paper/1912.02613/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Dec 2019.

## Built on by (most cited first, 18 of 18)

- [DiffSVC: A Diffusion Probabilistic Model for Singing Voice Conversion](https://mlsanity.com/paper/2105.13871/) (2021)
- [The Singing Voice Conversion Challenge 2023](https://mlsanity.com/paper/2306.14422/) (2023)
- [Hierarchical disentangled representation learning for singing voice conversion](https://mlsanity.com/paper/2101.06842/) (2021)
- [Towards High-fidelity Singing Voice Conversion with Acoustic Reference and Contrastive Predictive Coding](https://mlsanity.com/paper/2110.04754/) (2021)
- [Robust One-Shot Singing Voice Conversion](https://mlsanity.com/paper/2210.11096/) (2022)
- [Towards General-Purpose Text-Instruction-Guided Voice Conversion](https://mlsanity.com/paper/2309.14324/) (2023)
- [LDM-SVC: Latent Diffusion Model Based Zero-Shot Any-to-Any Singing Voice Conversion with Singer Guidance](https://mlsanity.com/paper/2406.05325/) (2024)
- [SaMoye: Zero-shot Singing Voice Conversion Model Based on Feature Disentanglement and Enhancement](https://mlsanity.com/paper/2407.07728/) (2024)
- [Recognizing Ornaments in Vocal Indian Art Music with Active Annotation](https://mlsanity.com/paper/2505.04419/) (2025)
- [HiFi-VC: High Quality ASR-Based Voice Conversion](https://mlsanity.com/paper/2203.16937/) (2022)
- [SingVisio: Visual Analytics of Diffusion Model for Singing Voice Conversion](https://mlsanity.com/paper/2402.12660/) (2024)
- [LCM-SVC: Latent Diffusion Model Based Singing Voice Conversion with Inference Acceleration via Latent Consistency Distillation](https://mlsanity.com/paper/2408.12354/) (2024)
- [DAFMSVC: One-Shot Singing Voice Conversion with Dual Attention Mechanism and Flow Matching](https://mlsanity.com/paper/2508.05978/) (2025)
- [A Comparative Analysis Of Latent Regressor Losses For Singing Voice Conversion](https://mlsanity.com/paper/2302.13678/) (2023)
- [Singing Voice Conversion with Accompaniment Using Self-Supervised Representation-Based Melody Features](https://mlsanity.com/paper/2502.04722/) (2025)
- [VibE-SVC: Vibrato Extraction with High-frequency F0 Contour for Singing Voice Conversion](https://mlsanity.com/paper/2505.20794/) (2025)
- [Effects of Convolutional Autoencoder Bottleneck Width on StarGAN-based Singing Technique Conversion](https://mlsanity.com/paper/2308.10021/) (2023)
- [S2Cap: A Benchmark and a Baseline for Singing Style Captioning](https://mlsanity.com/paper/2409.09866/) (2024)

## Abstract (arXiv)

We propose a flexible framework that deals with both singer conversion and singers vocal technique conversion. The proposed model is trained on non-parallel corpora, accommodates many-to-many conversion, and leverages recent advances of variational autoencoders. It employs separate encoders to learn disentangled latent representations of singer identity and vocal technique separately, with a joint decoder for reconstruction. Conversion is carried out by simple vector arithmetic in the learned latent spaces. Both a quantitative analysis as well as a visualization of the converted spectrograms show that our model is able to disentangle singer identity and vocal technique and successfully perform conversion of these attributes. To the best of our knowledge, this is the first work to jointly tackle conversion of singer identity and vocal technique based on a deep learning approach.