# VNet: A GAN-based Multi-Tier Discriminator Network for Speech Synthesis Vocoders

arXiv: https://arxiv.org/abs/2408.06906 · HTML: https://mlsanity.com/paper/2408.06906/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 32 references that are in the map.
- First posted on arXiv 13 Aug 2024.

## Builds on (most cited first, 25 of 25)

- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis](https://mlsanity.com/paper/2010.05646/) (2020)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [Conditional Image Generation with PixelCNN Decoders](https://mlsanity.com/paper/1606.05328/) (2016)
- [FastSpeech 2: Fast and High-Quality End-to-End Text to Speech](https://mlsanity.com/paper/2006.04558/) (2020)
- [LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech](https://mlsanity.com/paper/1904.02882/) (2019)
- [WaveGrad: Estimating Gradients for Waveform Generation](https://mlsanity.com/paper/2009.00713/) (2020)
- [Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech](https://mlsanity.com/paper/2105.06337/) (2021)
- [MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis](https://mlsanity.com/paper/1910.06711/) (2019)
- [Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram](https://mlsanity.com/paper/1910.11480/) (2019)
- [WaveGlow: A Flow-based Generative Network for Speech Synthesis](https://mlsanity.com/paper/1811.00002/) (2018)
- [BigVGAN: A Universal Neural Vocoder with Large-Scale Training](https://mlsanity.com/paper/2206.04658/) (2022)
- [Efficient Neural Audio Synthesis](https://mlsanity.com/paper/1802.08435/) (2018)
- [ProDiff: Progressive Fast Diffusion Model For High-Quality Text-to-Speech](https://mlsanity.com/paper/2207.06389/) (2022)
- [FastDiff: A Fast Conditional Diffusion Model for High-Quality Speech Synthesis](https://mlsanity.com/paper/2204.09934/) (2022)
- [ClariNet: Parallel Wave Generation in End-to-End Text-to-Speech](https://mlsanity.com/paper/1807.07281/) (2018)
- [Chunked Autoregressive GAN for Conditional Waveform Synthesis](https://mlsanity.com/paper/2110.10139/) (2021)
- [StyleMelGAN: An Efficient High-Fidelity Adversarial Vocoder with Temporal Adaptive Normalization](https://mlsanity.com/paper/2011.01557/) (2020)
- [JDI-T: Jointly trained Duration Informed Transformer for Text-To-Speech without Explicit Alignment](https://mlsanity.com/paper/2005.07799/) (2020)
- [Generative adversarial network-based glottal waveform model for statistical parametric speech synthesis](https://mlsanity.com/paper/1903.05955/) (2019)
- [LVCNet: Efficient Condition-Dependent Modeling Network for Waveform Generation](https://mlsanity.com/paper/2102.10815/) (2021)
- [BigVSAN: Enhancing GAN-based Neural Vocoders with Slicing Adversarial Network](https://mlsanity.com/paper/2309.02836/) (2023)
- [MFHCA: Enhancing Speech Emotion Recognition Via Multi-Spatial Fusion and Hierarchical Cooperative Attention](https://mlsanity.com/paper/2404.13509/) (2024)
- [Rethinking Gradient Operator for Exposing AI-enabled Face Forgeries](https://mlsanity.com/paper/2205.00767/) (2022)

## Built on by (most cited first, 10 of 10)

- [CIS-BWE: Chaos-Informed Speech Bandwidth Extension](https://mlsanity.com/paper/2507.15970/) (2025)
- [EAD-Net: Emotion-Aware Talking Head Generation with Spatial Refinement and Temporal Coherence](https://mlsanity.com/paper/2604.23325/) (2026)
- [ML-SAN: Multi-Level Speaker-Adaptive Network for Emotion Recognition in Conversations](https://mlsanity.com/paper/2604.25383/) (2026)
- [MRUF: Multi-granularity Routing with Uncertainty-Aware Fusion for Robust Multimodal Sentiment Analysis](https://mlsanity.com/paper/2607.10599/) (2026)
- [HRO: Hierarchical Room-to-Object Framework for Zero-Shot Object Goal Navigation with Large Language Models](https://mlsanity.com/paper/2607.13072/) (2026)
- [CTAN: Cycle-Temporal Attention Network for Embodied Audio-Visual Navigation](https://mlsanity.com/paper/2609.17420/) (2026)
- [Transformer-Based Token Fusion and Dynamic Graph Planning for Audio-Visual Navigation](https://mlsanity.com/paper/2609.17421/) (2026)
- [Talking Head Synthesis with Facial Landmark Guidance via 3D Gaussian Splatting](https://mlsanity.com/paper/2609.17422/) (2026)
- [WS-NeRF: A Mamba-Driven World-State-Aware Adaptive Deblurring Neural Radiance Field](https://mlsanity.com/paper/2609.21391/) (2026)
- [Two-Stage Multi-modal Learning for Dynamic Audio-Visual Navigation: From Physical Alignment to Scene Aggregation](https://mlsanity.com/paper/2610.14077/) (2026)

## Abstract (arXiv)

Since the introduction of Generative Adversarial Networks (GANs) in speech synthesis, remarkable achievements have been attained. In a thorough exploration of vocoders, it has been discovered that audio waveforms can be generated at speeds exceeding real-time while maintaining high fidelity, achieved through the utilization of GAN-based models. Typically, the inputs to the vocoder consist of band-limited spectral information, which inevitably sacrifices high-frequency details. To address this, we adopt the full-band Mel spectrogram information as input, aiming to provide the vocoder with the most comprehensive information possible. However, previous studies have revealed that the use of full-band spectral information as input can result in the issue of over-smoothing, compromising the naturalness of the synthesized speech. To tackle this challenge, we propose VNet, a GAN-based neural vocoder network that incorporates full-band spectral information and introduces a Multi-Tier Discriminator (MTD) comprising multiple sub-discriminators to generate high-resolution signals. Additionally, we introduce an asymptotically constrained method that modifies the adversarial loss of the generator and discriminator, enhancing the stability of the training process. Through rigorous experiments, we demonstrate that the VNet model is capable of generating high-fidelity speech and significantly improving the performance of the vocoder.