# TGAVC: Improving Autoencoder Voice Conversion with Text-Guided and Adversarial Training

arXiv: https://arxiv.org/abs/2208.04035 · HTML: https://mlsanity.com/paper/2208.04035/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 22 references that are in the map.
- First posted on arXiv 8 Aug 2022.

## Builds on (most cited first, 13 of 13)

- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [FastSpeech 2: Fast and High-Quality End-to-End Text to Speech](https://mlsanity.com/paper/2006.04558/) (2020)
- [Generalized End-to-End Loss for Speaker Verification](https://mlsanity.com/paper/1710.10467/) (2017)
- [AUTOVC: Zero-Shot Voice Style Transfer with Only Autoencoder Loss](https://mlsanity.com/paper/1905.05879/) (2019)
- [Unsupervised Learning of Disentangled and Interpretable Representations from Sequential Data](https://mlsanity.com/paper/1709.07902/) (2017)
- [Unsupervised speech representation learning using WaveNet autoencoders](https://mlsanity.com/paper/1901.08810/) (2019)
- [StarGAN-VC: Non-parallel many-to-many voice conversion with star generative adversarial networks](https://mlsanity.com/paper/1806.02169/) (2018)
- [Deep Voice 2: Multi-Speaker Neural Text-to-Speech](https://mlsanity.com/paper/1705.08947/) (2017)
- [CycleGAN-VC2: Improved CycleGAN-based Non-parallel Voice Conversion](https://mlsanity.com/paper/1904.04631/) (2019)
- [Voice Conversion from Non-parallel Corpora Using Variational Auto-encoder](https://mlsanity.com/paper/1610.04019/) (2016)
- [VQVC+: One-Shot Voice Conversion by Vector Quantization and U-Net architecture](https://mlsanity.com/paper/2006.04154/) (2020)
- [Blow: a single-scale hyperconditioned flow for non-parallel raw-audio voice conversion](https://mlsanity.com/paper/1906.00794/) (2019)
- [GAZEV: GAN-Based Zero-Shot Voice Conversion over Non-parallel Speech Corpus](https://mlsanity.com/paper/2010.12788/) (2020)

## Built on by (most cited first, 4 of 4)

- [Pose Guided Human Image Synthesis with Partially Decoupled GAN](https://mlsanity.com/paper/2210.03627/) (2022)
- [Semi-Supervised Learning Based on Reference Model for Low-resource TTS](https://mlsanity.com/paper/2210.14723/) (2022)
- [Boosting Star-GANs for Voice Conversion with Contrastive Discriminator](https://mlsanity.com/paper/2209.10088/) (2022)
- [MetaSpeech: Speech Effects Switch Along with Environment for Metaverse](https://mlsanity.com/paper/2210.13811/) (2022)

## Abstract (arXiv)

Non-parallel many-to-many voice conversion remains an interesting but challenging speech processing task. Recently, AutoVC, a conditional autoencoder based method, achieved excellent conversion results by disentangling the speaker identity and the speech content using information-constraining bottlenecks. However, due to the pure autoencoder training method, it is difficult to evaluate the separation effect of content and speaker identity. In this paper, a novel voice conversion framework, named boldsymbol Text boldsymbol Guided boldsymbol AutoVC(TGAVC), is proposed to more effectively separate content and timbre from speech, where an expected content embedding produced based on the text transcriptions is designed to guide the extraction of voice content. In addition, the adversarial training is applied to eliminate the speaker identity information in the estimated content embedding extracted from speech. Under the guidance of the expected content embedding and the adversarial training, the content encoder is trained to extract speaker-independent content embedding from speech. Experiments on AIShell-3 dataset show that the proposed model outperforms AutoVC in terms of naturalness and similarity of converted speech.