# Fre-GAN: Adversarial Frequency-consistent Audio Synthesis

arXiv: https://arxiv.org/abs/2106.02297 · HTML: https://mlsanity.com/paper/2106.02297/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 33 references that are in the map.
- First posted on arXiv 4 Jun 2021.

## Builds on (most cited first, 23 of 23)

- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis](https://mlsanity.com/paper/2010.05646/) (2020)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [Autoencoding beyond pixels using a learned similarity metric](https://mlsanity.com/paper/1512.09300/) (2015)
- [Making Convolutional Networks Shift-Invariant Again](https://mlsanity.com/paper/1904.11486/) (2019)
- [MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis](https://mlsanity.com/paper/1910.06711/) (2019)
- [Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram](https://mlsanity.com/paper/1910.11480/) (2019)
- [WaveGlow: A Flow-based Generative Network for Speech Synthesis](https://mlsanity.com/paper/1811.00002/) (2018)
- [Efficient Neural Audio Synthesis](https://mlsanity.com/paper/1802.08435/) (2018)
- [High Fidelity Speech Synthesis with Adversarial Networks](https://mlsanity.com/paper/1909.11646/) (2019)
- [FloWaveNet : A Generative Flow for Raw Audio](https://mlsanity.com/paper/1811.02155/) (2018)
- [SSD-GAN: Measuring the Realness in the Spatial and Spectral Domains](https://mlsanity.com/paper/2012.05535/) (2020)
- [Upsampling artifacts in neural audio synthesis](https://mlsanity.com/paper/2010.14356/) (2020)
- [Multi-SpectroGAN: High-Diversity and High-Fidelity Spectrogram Generation with Adversarial Style Combination for Speech Synthesis](https://mlsanity.com/paper/2012.07267/) (2020)
- [VocGAN: A High-Fidelity Real-time Vocoder with a Hierarchically-nested Adversarial Network](https://mlsanity.com/paper/2007.15256/) (2020)
- [Universal MelGAN: A Robust Neural Vocoder for High-Fidelity Waveform Generation in Multiple Domains](https://mlsanity.com/paper/2011.09631/) (2020)
- [Audio Dequantization for High Fidelity Audio Generation in Flow-based Neural Vocoder](https://mlsanity.com/paper/2008.06867/) (2020)

## Built on by (most cited first, 24 of 24)

- [A Survey on Neural Speech Synthesis](https://mlsanity.com/paper/2106.15561/) (2021)
- [Recent Advances in Speech Language Models: A Survey](https://mlsanity.com/paper/2410.03751/) (2024)
- [HierSpeech++: Bridging the Gap between Semantic and Acoustic Representation of Speech by Hierarchical Variational Inference for Zero-shot Speech Synthesis](https://mlsanity.com/paper/2311.12454/) (2023)
- [Avocodo: Generative Adversarial Network for Artifact-free Vocoder](https://mlsanity.com/paper/2206.13404/) (2022)
- [Let There Be Sound: Reconstructing High Quality Speech from Silent Videos](https://mlsanity.com/paper/2308.15256/) (2023)
- [FreGrad: Lightweight and Fast Frequency-aware Diffusion Vocoder](https://mlsanity.com/paper/2401.10032/) (2024)
- [Accelerating Codec-based Speech Synthesis with Multi-Token Prediction and Speculative Decoding](https://mlsanity.com/paper/2410.13839/) (2024)
- [HierVST: Hierarchical Adaptive Zero-shot Voice Style Transfer](https://mlsanity.com/paper/2307.16171/) (2023)
- [PhaseAug: A Differentiable Augmentation for Speech Synthesis to Simulate One-to-Many Mapping](https://mlsanity.com/paper/2211.04610/) (2022)
- [WaveFM: A High-Fidelity and Efficient Vocoder Based on Flow Matching](https://mlsanity.com/paper/2503.16689/) (2025)
- [PauseSpeech: Natural Speech Synthesis via Pre-trained Language Model and Pause-based Prosody Modeling](https://mlsanity.com/paper/2306.07489/) (2023)
- [iSTFTNet2: Faster and More Lightweight iSTFT-Based Neural Vocoder Using 1D-2D CNN](https://mlsanity.com/paper/2308.07117/) (2023)
- [Accelerating High-Fidelity Waveform Generation via Adversarial Flow Matching Optimization](https://mlsanity.com/paper/2408.08019/) (2024)
- [CrossSpeech: Speaker-independent Acoustic Representation for Cross-lingual Speech Synthesis](https://mlsanity.com/paper/2302.14370/) (2023)
- [AV-GAN: Attention-Based Varifocal Generative Adversarial Network for Uneven Medical Image Translation](https://mlsanity.com/paper/2404.10714/) (2024)
- [SINGER: Vivid Audio-driven Singing Video Generation with Multi-scale Spectral Diffusion Model](https://mlsanity.com/paper/2412.03430/) (2024)
- [A Factorization Approach for Motor Imagery Classification](https://mlsanity.com/paper/2112.08175/) (2021)
- [An End-to-End Multi-Module Audio Deepfake Generation System for ADD Challenge 2023](https://mlsanity.com/paper/2307.00729/) (2023)
- [Training Generative Adversarial Network-Based Vocoder with Limited Data Using Augmentation-Conditional Discriminator](https://mlsanity.com/paper/2403.16464/) (2024)
- [Dataset Refinement for Improving the Generalization Ability of the EEG Decoding Model](https://mlsanity.com/paper/2411.10450/) (2024)
- [CrossSpeech++: Cross-lingual Speech Synthesis with Decoupled Language and Speaker Generation](https://mlsanity.com/paper/2412.20048/) (2024)
- [DPN-GAN: Inducing Periodic Activations in Generative Adversarial Networks for High-Fidelity Audio Synthesis](https://mlsanity.com/paper/2505.09091/) (2025)
- [CIS-BWE: Chaos-Informed Speech Bandwidth Extension](https://mlsanity.com/paper/2507.15970/) (2025)
- [ClipTBP: Clip-Pair based Temporal Boundary Prediction with Boundary-Aware Learning for Moment Retrieval](https://mlsanity.com/paper/2604.27591/) (2026)

## Abstract (arXiv)

Although recent works on neural vocoder have improved the quality of synthesized audio, there still exists a gap between generated and ground-truth audio in frequency space. This difference leads to spectral artifacts such as hissing noise or reverberation, and thus degrades the sample quality. In this paper, we propose Fre-GAN which achieves frequency-consistent audio synthesis with highly improved generation quality. Specifically, we first present resolution-connected generator and resolution-wise discriminators, which help learn various scales of spectral distributions over multiple frequency bands. Additionally, to reproduce high-frequency components accurately, we leverage discrete wavelet transform in the discriminators. From our experiments, Fre-GAN achieves high-fidelity waveform generation with a gap of only 0.03 MOS compared to ground-truth audio while outperforming standard models in quality.