# Glow-WaveGAN 2: High-quality Zero-shot Text-to-speech Synthesis and Any-to-any Voice Conversion

arXiv: https://arxiv.org/abs/2207.01832 · HTML: https://mlsanity.com/paper/2207.01832/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jul 2022.

## Built on by (most cited first, 5 of 5)

- [Fake the Real: Backdoor Attack on Deep Speech Classification via Voice Conversion](https://mlsanity.com/paper/2306.15875/) (2023)
- [Generalizable Zero-Shot Speaker Adaptive Speech Synthesis with Disentangled Representations](https://mlsanity.com/paper/2308.13007/) (2023)
- [Voice Cloning: Comprehensive Survey](https://mlsanity.com/paper/2505.00579/) (2025)
- [UnifySpeech: A Unified Framework for Zero-shot Text-to-Speech and Voice Conversion](https://mlsanity.com/paper/2301.03801/) (2023)
- [Automatic Tuning of Loss Trade-offs without Hyper-parameter Search in End-to-End Zero-Shot Speech Synthesis](https://mlsanity.com/paper/2305.16699/) (2023)

## Abstract (arXiv)

The zero-shot scenario for speech generation aims at synthesizing a novel unseen voice with only one utterance of the target speaker. Although the challenges of adapting new voices in zero-shot scenario exist in both stages -- acoustic modeling and vocoder, previous works usually consider the problem from only one stage. In this paper, we extend our previous Glow-WaveGAN to Glow-WaveGAN 2, aiming to solve the problem from both stages for high-quality zero-shot text-to-speech and any-to-any voice conversion. We first build a universal WaveGAN model for extracting latent distribution p(z) of speech and reconstructing waveform from it. Then a flow-based acoustic model only needs to learn the same p(z) from texts, which naturally avoids the mismatch between the acoustic model and the vocoder, resulting in high-quality generated speech without model fine-tuning. Based on a continuous speaker space and the reversible property of flows, the conditional distribution can be obtained for any speaker, and thus we can further conduct high-quality zero-shot speech generation for new speakers. We particularly investigate two methods to construct the speaker space, namely pre-trained speaker encoder and jointly-trained speaker encoder. The superiority of Glow-WaveGAN 2 has been proved through TTS and VC experiments conducted on LibriTTS corpus and VTCK corpus.