# VocGAN: A High-Fidelity Real-time Vocoder with a Hierarchically-nested Adversarial Network

arXiv: https://arxiv.org/abs/2007.15256 · HTML: https://mlsanity.com/paper/2007.15256/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Jul 2020.

## Built on by (most cited first, 26 of 26)

- [A Survey on Neural Speech Synthesis](https://mlsanity.com/paper/2106.15561/) (2021)
- [WaveGrad 2: Iterative Refinement for Text-to-Speech Synthesis](https://mlsanity.com/paper/2106.09660/) (2021)
- [DiffGAN-TTS: High-Fidelity and Efficient Text-to-Speech with Denoising Diffusion GANs](https://mlsanity.com/paper/2201.11972/) (2022)
- [iSTFTNet: Fast and Lightweight Mel-Spectrogram Vocoder Incorporating Inverse Short-Time Fourier Transform](https://mlsanity.com/paper/2203.02395/) (2022)
- [Fre-GAN: Adversarial Frequency-consistent Audio Synthesis](https://mlsanity.com/paper/2106.02297/) (2021)
- [GANSpeech: Adversarial Training for High-Fidelity Multi-Speaker Speech Synthesis](https://mlsanity.com/paper/2106.15153/) (2021)
- [Avocodo: Generative Adversarial Network for Artifact-free Vocoder](https://mlsanity.com/paper/2206.13404/) (2022)
- [GAN Vocoder: Multi-Resolution Discriminator Is All You Need](https://mlsanity.com/paper/2103.05236/) (2021)
- [EmoSpeech: Guiding FastSpeech2 Towards Emotional Text to Speech](https://mlsanity.com/paper/2307.00024/) (2023)
- [FastPitchFormant: Source-filter based Decomposed Modeling for Speech Synthesis](https://mlsanity.com/paper/2106.15123/) (2021)
- [PhaseAug: A Differentiable Augmentation for Speech Synthesis to Simulate One-to-Many Mapping](https://mlsanity.com/paper/2211.04610/) (2022)
- [Review of end-to-end speech synthesis technology based on deep learning](https://mlsanity.com/paper/2104.09995/) (2021)
- [Neural Synthesis of Footsteps Sound Effects with Generative Adversarial Networks](https://mlsanity.com/paper/2110.09605/) (2021)
- [Wave-U-Net Discriminator: Fast and Lightweight Discriminator for Generative Adversarial Network-Based Speech Synthesis](https://mlsanity.com/paper/2303.13909/) (2023)
- [iSTFTNet2: Faster and More Lightweight iSTFT-Based Neural Vocoder Using 1D-2D CNN](https://mlsanity.com/paper/2308.07117/) (2023)
- [Unified Source-Filter GAN: Unified Source-filter Network Based On Factorization of Quasi-Periodic Parallel WaveGAN](https://mlsanity.com/paper/2104.04668/) (2021)
- [cMelGAN: An Efficient Conditional Generative Model Based on Mel Spectrograms](https://mlsanity.com/paper/2205.07319/) (2022)
- [MnTTS: An Open-Source Mongolian Text-to-Speech Synthesis Dataset and Accompanied Baseline](https://mlsanity.com/paper/2209.10848/) (2022)
- [Improve GAN-based Neural Vocoder using Pointwise Relativistic LeastSquare GAN](https://mlsanity.com/paper/2103.14245/) (2021)
- [UniTTS: Residual Learning of Unified Embedding Space for Speech Style Control](https://mlsanity.com/paper/2106.11171/) (2021)
- [DeepA: A Deep Neural Analyzer For Speech And Singing Vocoding](https://mlsanity.com/paper/2110.06434/) (2021)
- [Enhancement of Pitch Controllability using Timbre-Preserving Pitch Augmentation in FastPitch](https://mlsanity.com/paper/2204.05753/) (2022)
- [MSR-NV: Neural Vocoder Using Multiple Sampling Rates](https://mlsanity.com/paper/2109.13714/) (2021)
- [WOLONet: Wave Outlooker for Efficient and High Fidelity Speech Synthesis](https://mlsanity.com/paper/2206.09920/) (2022)
- [Style Description based Text-to-Speech with Conditional Prosodic Layer Normalization based Diffusion GAN](https://mlsanity.com/paper/2310.18169/) (2023)
- [Training Generative Adversarial Network-Based Vocoder with Limited Data Using Augmentation-Conditional Discriminator](https://mlsanity.com/paper/2403.16464/) (2024)

## Abstract (arXiv)

We present a novel high-fidelity real-time neural vocoder called VocGAN. A recently developed GAN-based vocoder, MelGAN, produces speech waveforms in real-time. However, it often produces a waveform that is insufficient in quality or inconsistent with acoustic characteristics of the input mel spectrogram. VocGAN is nearly as fast as MelGAN, but it significantly improves the quality and consistency of the output waveform. VocGAN applies a multi-scale waveform generator and a hierarchically-nested discriminator to learn multiple levels of acoustic properties in a balanced way. It also applies the joint conditional and unconditional objective, which has shown successful results in high-resolution image synthesis. In experiments, VocGAN synthesizes speech waveforms 416.7x faster on a GTX 1080Ti GPU and 3.24x faster on a CPU than real-time. Compared with MelGAN, it also exhibits significantly improved quality in multiple evaluation metrics including mean opinion score (MOS) with minimal additional overhead. Additionally, compared with Parallel WaveGAN, another recently developed high-fidelity vocoder, VocGAN is 6.98x faster on a CPU and exhibits higher MOS.