# Many-to-Many Voice Conversion based Feature Disentanglement using Variational Autoencoder

arXiv: https://arxiv.org/abs/2107.06642 · HTML: https://mlsanity.com/paper/2107.06642/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 27 references that are in the map.
- First posted on arXiv 11 Jul 2021.

## Builds on (most cited first, 19 of 19)

- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions](https://mlsanity.com/paper/1712.05884/) (2017)
- [Semi-Supervised Learning with Deep Generative Models](https://mlsanity.com/paper/1406.5298/) (2014)
- [Weakly-Supervised Disentanglement Without Compromises](https://mlsanity.com/paper/2002.02886/) (2020)
- [AUTOVC: Zero-Shot Voice Style Transfer with Only Autoencoder Loss](https://mlsanity.com/paper/1905.05879/) (2019)
- [Disentangled Sequential Autoencoder](https://mlsanity.com/paper/1803.02991/) (2018)
- [Unsupervised Learning of Disentangled and Interpretable Representations from Sequential Data](https://mlsanity.com/paper/1709.07902/) (2017)
- [CycleGAN-VC2: Improved CycleGAN-based Non-parallel Voice Conversion](https://mlsanity.com/paper/1904.04631/) (2019)
- [Voice Conversion from Non-parallel Corpora Using Variational Auto-encoder](https://mlsanity.com/paper/1610.04019/) (2016)
- [StarGAN-VC2: Rethinking Conditional Methods for StarGAN-Based Voice Conversion](https://mlsanity.com/paper/1907.12279/) (2019)
- [F0-consistent many-to-many non-parallel voice conversion via conditional autoencoder](https://mlsanity.com/paper/2004.07370/) (2020)
- [VQVC+: One-Shot Voice Conversion by Vector Quantization and U-Net architecture](https://mlsanity.com/paper/2006.04154/) (2020)
- [Improving Zero-shot Voice Style Transfer via Disentangled Representation Learning](https://mlsanity.com/paper/2103.09420/) (2021)
- [Multi-target Voice Conversion without Parallel Data by Adversarially Learning Disentangled Audio Representations](https://mlsanity.com/paper/1804.02812/) (2018)
- [Non-Parallel Voice Conversion with Cyclic Variational Autoencoder](https://mlsanity.com/paper/1907.10185/) (2019)
- [ACVAE-VC: Non-parallel many-to-many voice conversion with auxiliary classifier variational autoencoder](https://mlsanity.com/paper/1808.05092/) (2018)
- [Many-to-Many Voice Conversion using Conditional Cycle-Consistent Adversarial Networks](https://mlsanity.com/paper/2002.06328/) (2020)
- [Baseline System of Voice Conversion Challenge 2020 with Cyclic Variational Autoencoder and Parallel WaveGAN](https://mlsanity.com/paper/2010.04429/) (2020)

## Built on by (most cited first, 7 of 7)

- [Diffusion-Based Voice Conversion with Fast Maximum Likelihood Sampling Scheme](https://mlsanity.com/paper/2109.13821/) (2021)
- [Training Robust Zero-Shot Voice Conversion Models with Self-supervised Features](https://mlsanity.com/paper/2112.04424/) (2021)
- [DSVAE: Interpretable Disentangled Representation for Synthetic Speech Detection](https://mlsanity.com/paper/2304.03323/) (2023)
- [Improving Generalization for AI-Synthesized Voice Detection](https://mlsanity.com/paper/2412.19279/) (2024)
- [Prosody-Adaptable Audio Codecs for Zero-Shot Voice Conversion via In-Context Learning](https://mlsanity.com/paper/2505.15402/) (2025)
- [Enhancing Zero-Shot Many to Many Voice Conversion with Self-Attention VAE](https://mlsanity.com/paper/2203.16037/) (2022)
- [Automatic Speech Disentanglement for Voice Conversion using Rank Module and Speech Augmentation](https://mlsanity.com/paper/2306.12259/) (2023)

## Abstract (arXiv)

Voice conversion is a challenging task which transforms the voice characteristics of a source speaker to a target speaker without changing linguistic content. Recently, there have been many works on many-to-many Voice Conversion (VC) based on Variational Autoencoder (VAEs) achieving good results, however, these methods lack the ability to disentangle speaker identity and linguistic content to achieve good performance on unseen speaker scenarios. In this paper, we propose a new method based on feature disentanglement to tackle many to many voice conversion. The method has the capability to disentangle speaker identity and linguistic content from utterances, it can convert from many source speakers to many target speakers with a single autoencoder network. Moreover, it naturally deals with the unseen target speaker scenarios. We perform both objective and subjective evaluations to show the competitive performance of our proposed method compared with other state-of-the-art models in terms of naturalness and target speaker similarity.