# Multi-band MelGAN: Faster Waveform Generation for High-Quality Text-to-Speech

arXiv: https://arxiv.org/abs/2005.05106 · HTML: https://mlsanity.com/paper/2005.05106/

## Key facts

- Cited by 63 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 May 2020.

## Built on by (most cited first, 30 of 63)

- [High-Fidelity Audio Compression with Improved RVQGAN](https://mlsanity.com/paper/2306.06546/) (2023)
- [BigVGAN: A Universal Neural Vocoder with Large-Scale Training](https://mlsanity.com/paper/2206.04658/) (2022)
- [A Survey on Neural Speech Synthesis](https://mlsanity.com/paper/2106.15561/) (2021)
- [WaveFake: A Data Set to Facilitate Audio Deepfake Detection](https://mlsanity.com/paper/2111.02813/) (2021)
- [RAVE: A variational autoencoder for fast and high-quality neural audio synthesis](https://mlsanity.com/paper/2111.05011/) (2021)
- [WaveGrad 2: Iterative Refinement for Text-to-Speech Synthesis](https://mlsanity.com/paper/2106.09660/) (2021)
- [BDDM: Bilateral Denoising Diffusion Models for Fast and High-Quality Speech Synthesis](https://mlsanity.com/paper/2203.13508/) (2022)
- [SpecGrad: Diffusion Probabilistic Model based Neural Vocoder with Adaptive Noise Spectral Shaping](https://mlsanity.com/paper/2203.16749/) (2022)
- [Bilateral Denoising Diffusion Models](https://mlsanity.com/paper/2108.11514/) (2021)
- [SVTS: Scalable Video-to-Speech Synthesis](https://mlsanity.com/paper/2205.02058/) (2022)
- [iSTFTNet: Fast and Lightweight Mel-Spectrogram Vocoder Incorporating Inverse Short-Time Fourier Transform](https://mlsanity.com/paper/2203.02395/) (2022)
- [Towards Building Text-To-Speech Systems for the Next Billion Users](https://mlsanity.com/paper/2211.09536/) (2022)
- [AI-Generated Content (AIGC) for Various Data Modalities: A Survey](https://mlsanity.com/paper/2308.14177/) (2023)
- [ESPnet2-TTS: Extending the Edge of TTS Research](https://mlsanity.com/paper/2110.07840/) (2021)
- [SafeEar: Content Privacy-Preserving Audio Deepfake Detection](https://mlsanity.com/paper/2409.09272/) (2024)
- [WaveFit: An Iterative and Non-autoregressive Neural Vocoder based on Fixed-Point Iteration](https://mlsanity.com/paper/2210.01029/) (2022)
- [Avocodo: Generative Adversarial Network for Artifact-free Vocoder](https://mlsanity.com/paper/2206.13404/) (2022)
- [AnyoneNet: Synchronized Speech and Talking Head Generation for Arbitrary Person](https://mlsanity.com/paper/2108.04325/) (2021)
- [GAN Vocoder: Multi-Resolution Discriminator Is All You Need](https://mlsanity.com/paper/2103.05236/) (2021)
- [HUI-Audio-Corpus-German: A high quality TTS dataset](https://mlsanity.com/paper/2106.06309/) (2021)
- [An Initial Investigation for Detecting Vocoder Fingerprints of Fake Audio](https://mlsanity.com/paper/2208.09646/) (2022)
- [Modelling black-box audio effects with time-varying feature modulation](https://mlsanity.com/paper/2211.00497/) (2022)
- [Where are we in audio deepfake detection? A systematic analysis over generative and detection models](https://mlsanity.com/paper/2410.04324/) (2024)
- [SpecRNet: Towards Faster and More Accessible Audio DeepFake Detection](https://mlsanity.com/paper/2210.06105/) (2022)
- [Lightweight and High-Fidelity End-to-End Text-to-Speech with Multi-Band Generation and Inverse Short-Time Fourier Transform](https://mlsanity.com/paper/2210.15975/) (2022)
- [RFWave: Multi-band Rectified Flow for Audio Waveform Reconstruction](https://mlsanity.com/paper/2403.05010/) (2024)
- [VocBench: A Neural Vocoder Benchmark for Speech Synthesis](https://mlsanity.com/paper/2112.03099/) (2021)
- [RapVerse: Coherent Vocals and Whole-Body Motions Generations from Text](https://mlsanity.com/paper/2405.20336/) (2024)
- [Review of end-to-end speech synthesis technology based on deep learning](https://mlsanity.com/paper/2104.09995/) (2021)
- [A Streamwise GAN Vocoder for Wideband Speech Coding at Very Low Bit Rate](https://mlsanity.com/paper/2108.04051/) (2021)

## Abstract (arXiv)

In this paper, we propose multi-band MelGAN, a much faster waveform generation model targeting to high-quality text-to-speech. Specifically, we improve the original MelGAN by the following aspects. First, we increase the receptive field of the generator, which is proven to be beneficial to speech generation. Second, we substitute the feature matching loss with the multi-resolution STFT loss to better measure the difference between fake and real speech. Together with pre-training, this improvement leads to both better quality and better training stability. More importantly, we extend MelGAN with multi-band processing: the generator takes mel-spectrograms as input and produces sub-band signals which are subsequently summed back to full-band signals as discriminator input. The proposed multi-band MelGAN has achieved high MOS of 4.34 and 4.22 in waveform generation and TTS, respectively. With only 1.91M parameters, our model effectively reduces the total computational complexity of the original MelGAN from 5.85 to 0.95 GFLOPS. Our Pytorch implementation, which will be open-resourced shortly, can achieve a real-time factor of 0.03 on CPU without hardware specific optimization.