# Unsupervised Representation Disentanglement using Cross Domain Features and Adversarial Learning in Variational Autoencoder based Voice Conversion

arXiv: https://arxiv.org/abs/2001.07849 · HTML: https://mlsanity.com/paper/2001.07849/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Jan 2020.

## Built on by (most cited first, 13 of 13)

- [Deep Stable Learning for Out-Of-Distribution Generalization](https://mlsanity.com/paper/2104.07876/) (2021)
- [Deepfakes Generation and Detection: State-of-the-art, open challenges, countermeasures, and way forward](https://mlsanity.com/paper/2103.00484/) (2021)
- [StarGANv2-VC: A Diverse, Unsupervised, Non-parallel Framework for Natural-Sounding Voice Conversion](https://mlsanity.com/paper/2107.10394/) (2021)
- [Adversarially learning disentangled speech representations for robust multi-factor voice conversion](https://mlsanity.com/paper/2102.00184/) (2021)
- [An Improved StarGAN for Emotional Voice Conversion: Enhancing Voice Quality and Data Augmentation](https://mlsanity.com/paper/2107.08361/) (2021)
- [A Preliminary Study of a Two-Stage Paradigm for Preserving Speaker Identity in Dysarthric Voice Conversion](https://mlsanity.com/paper/2106.01415/) (2021)
- [Global Rhythm Style Transfer Without Text Transcriptions](https://mlsanity.com/paper/2106.08519/) (2021)
- [Reimagining Speech: A Scoping Review of Deep Learning-Powered Voice Conversion](https://mlsanity.com/paper/2311.08104/) (2023)
- [Non-Parallel Voice Conversion for ASR Augmentation](https://mlsanity.com/paper/2209.06987/) (2022)
- [Evaluation of Speaker Anonymization on Emotional Speech](https://mlsanity.com/paper/2305.01759/) (2023)
- [Advancing the dimensionality reduction of speaker embeddings for speaker diarisation: disentangling noise and informing speech activity](https://mlsanity.com/paper/2110.03380/) (2021)
- [Voice Conversion with Denoising Diffusion Probabilistic GAN Models](https://mlsanity.com/paper/2308.14319/) (2023)
- [Koopman Regularized Deep Speech Disentanglement for Speaker Verification](https://mlsanity.com/paper/2603.05577/) (2026)

## Abstract (arXiv)

An effective approach for voice conversion (VC) is to disentangle linguistic content from other components in the speech signal. The effectiveness of variational autoencoder (VAE) based VC (VAE-VC), for instance, strongly relies on this principle. In our prior work, we proposed a cross-domain VAE-VC (CDVAE-VC) framework, which utilized acoustic features of different properties, to improve the performance of VAE-VC. We believed that the success came from more disentangled latent representations. In this paper, we extend the CDVAE-VC framework by incorporating the concept of adversarial learning, in order to further increase the degree of disentanglement, thereby improving the quality and similarity of converted speech. More specifically, we first investigate the effectiveness of incorporating the generative adversarial networks (GANs) with CDVAE-VC. Then, we consider the concept of domain adversarial training and add an explicit constraint to the latent representation, realized by a speaker classifier, to explicitly eliminate the speaker information that resides in the latent code. Experimental results confirm that the degree of disentanglement of the learned latent representation can be enhanced by both GANs and the speaker classifier. Meanwhile, subjective evaluation results in terms of quality and similarity scores demonstrate the effectiveness of our proposed methods.