# ACVAE-VC: Non-parallel many-to-many voice conversion with auxiliary classifier variational autoencoder

arXiv: https://arxiv.org/abs/1808.05092 · HTML: https://mlsanity.com/paper/1808.05092/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Aug 2018.

## Built on by (most cited first, 9 of 9)

- [Many-to-Many Voice Conversion based Feature Disentanglement using Variational Autoencoder](https://mlsanity.com/paper/2107.06642/) (2021)
- [Wasserstein Geodesic Generator for Conditional Distributions](https://mlsanity.com/paper/2308.10145/) (2023)
- [Voicy: Zero-Shot Non-Parallel Voice Conversion in Noisy Reverberant Environments](https://mlsanity.com/paper/2106.08873/) (2021)
- [Covariate-informed Representation Learning to Prevent Posterior Collapse of iVAE](https://mlsanity.com/paper/2202.04206/) (2022)
- [Axial Residual Networks for CycleGAN-based Voice Conversion](https://mlsanity.com/paper/2102.08075/) (2021)
- [Conditional Deep Hierarchical Variational Autoencoder for Voice Conversion](https://mlsanity.com/paper/2112.02796/) (2021)
- [MoEVC: A Mixture-of-experts Voice Conversion System with Sparse Gating Mechanism for Accelerating Online Computation](https://mlsanity.com/paper/1912.11984/) (2019)
- [CycleFlow: Purify Information Factors by Cycle Loss](https://mlsanity.com/paper/2110.09928/) (2021)
- [MetaSpeech: Speech Effects Switch Along with Environment for Metaverse](https://mlsanity.com/paper/2210.13811/) (2022)

## Abstract (arXiv)

This paper proposes a non-parallel many-to-many voice conversion (VC) method using a variant of the conditional variational autoencoder (VAE) called an auxiliary classifier VAE (ACVAE). The proposed method has three key features. First, it adopts fully convolutional architectures to construct the encoder and decoder networks so that the networks can learn conversion rules that capture time dependencies in the acoustic feature sequences of source and target speech. Second, it uses an information-theoretic regularization for the model training to ensure that the information in the attribute class label will not be lost in the conversion process. With regular CVAEs, the encoder and decoder are free to ignore the attribute class label input. This can be problematic since in such a situation, the attribute class label will have little effect on controlling the voice characteristics of input speech at test time. Such situations can be avoided by introducing an auxiliary classifier and training the encoder and decoder so that the attribute classes of the decoder outputs are correctly predicted by the classifier. Third, it avoids producing buzzy-sounding speech at test time by simply transplanting the spectral details of the input speech into its converted version. Subjective evaluation experiments revealed that this simple method worked reasonably well in a non-parallel many-to-many speaker identity conversion task.