# TFGAN: Time and Frequency Domain Based Generative Adversarial Network for High-fidelity Speech Synthesis

arXiv: https://arxiv.org/abs/2011.12206 · HTML: https://mlsanity.com/paper/2011.12206/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Nov 2020.

## Built on by (most cited first, 11 of 11)

- [HiFi++: a Unified Framework for Bandwidth Extension and Speech Enhancement](https://mlsanity.com/paper/2203.13086/) (2022)
- [Neural Vocoder is All You Need for Speech Super-resolution](https://mlsanity.com/paper/2203.14941/) (2022)
- [Audio Super-Resolution with Latent Bridge Models](https://mlsanity.com/paper/2509.17609/) (2025)
- [Modelling black-box audio effects with time-varying feature modulation](https://mlsanity.com/paper/2211.00497/) (2022)
- [VoiceShop: A Unified Speech-to-Speech Framework for Identity-Preserving Zero-Shot Voice Editing](https://mlsanity.com/paper/2404.06674/) (2024)
- [Advancing Talking Head Generation: A Comprehensive Survey of Multi-Modal Methodologies, Datasets, Evaluation Metrics, and Loss Functions](https://mlsanity.com/paper/2507.02900/) (2025)
- [WaveletDiff: Multilevel Wavelet Diffusion For Time Series Generation](https://mlsanity.com/paper/2510.11839/) (2025)
- [Improve GAN-based Neural Vocoder using Pointwise Relativistic LeastSquare GAN](https://mlsanity.com/paper/2103.14245/) (2021)
- [A High-Fidelity Speech Super Resolution Network using a Complex Global Attention Module with Spectro-Temporal Loss](https://mlsanity.com/paper/2507.00229/) (2025)
- [SUBARU: A Practical Approach to Power Saving in Hearables Using SUB-Nyquist Audio Resolution Upsampling](https://mlsanity.com/paper/2506.22321/) (2025)
- [CIS-BWE: Chaos-Informed Speech Bandwidth Extension](https://mlsanity.com/paper/2507.15970/) (2025)

## Abstract (arXiv)

Recently, GAN based speech synthesis methods, such as MelGAN, have become very popular. Compared to conventional autoregressive based methods, parallel structures based generators make waveform generation process fast and stable. However, the quality of generated speech by autoregressive based neural vocoders, such as WaveRNN, is still higher than GAN. To address this issue, we propose a novel vocoder model: TFGAN, which is adversarially learned both in time and frequency domain. On one hand, we propose to discriminate ground-truth waveform from synthetic one in frequency domain for offering more consistency guarantees instead of only in time domain. On the other hand, in contrast to the conventionally frequency-domain STFT loss approach or feature map loss by discriminator to learn waveform, we propose a set of time-domain loss that encourage the generator to capture the waveform directly. TFGAN has nearly same synthesis speed as MelGAN, but the fidelity is significantly improved by our novel learning method. In our experiments, TFGAN shows the ability to achieve comparable mean opinion score (MOS) than autoregressive vocoder under speech synthesis context.