mlsanityData to 7 Oct 2026

Multi-band MelGAN: Faster Waveform Generation for High-Quality Text-to-Speech

arXiv 2005.05106 ↗First posted 11 May 2020cs.SD eess.AScited by 63

Not on the map. Why?

Abstract (arXiv)

In this paper, we propose multi-band MelGAN, a much faster waveform generation model targeting to high-quality text-to-speech. Specifically, we improve the original MelGAN by the following aspects. First, we increase the receptive field of the generator, which is proven to be beneficial to speech generation. Second, we substitute the feature matching loss with the multi-resolution STFT loss to better measure the difference between fake and real speech. Together with pre-training, this improvement leads to both better quality and better training stability. More importantly, we extend MelGAN with multi-band processing: the generator takes mel-spectrograms as input and produces sub-band signals which are subsequently summed back to full-band signals as discriminator input. The proposed multi-band MelGAN has achieved high MOS of 4.34 and 4.22 in waveform generation and TTS, respectively. With only 1.91M parameters, our model effectively reduces the total computational complexity of the original MelGAN from 5.85 to 0.95 GFLOPS. Our Pytorch implementation, which will be open-resourced shortly, can achieve a real-time factor of 0.03 on CPU without hardware specific optimization.

Key facts. Cited by 63 papers in the map (data through 7 Oct 2026). First posted on arXiv 11 May 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 19; 2022: 15; 2023: 9; 2024: 8; 2025: 8; 2026: 4.

63 papers.

Sort:

By area:

  1. 2023High-Fidelity Audio Compression with Improved RVQGANcited by 423
  2. 2022BigVGAN: A Universal Neural Vocoder with Large-Scale Trainingcited by 245
  3. 2021A Survey on Neural Speech Synthesiscited by 182
  4. 2021WaveFake: A Data Set to Facilitate Audio Deepfake Detectioncited by 78
  5. 2021RAVE: A variational autoencoder for fast and high-quality neural audio synthesiscited by 72
  6. 2021WaveGrad 2: Iterative Refinement for Text-to-Speech Synthesiscited by 65
  7. 2022BDDM: Bilateral Denoising Diffusion Models for Fast and High-Quality Speech Synthesiscited by 64
  8. 2022SpecGrad: Diffusion Probabilistic Model based Neural Vocoder with Adaptive Noise Spectral Shapingcited by 37
  9. 2021Bilateral Denoising Diffusion Modelscited by 36
  10. 2022SVTS: Scalable Video-to-Speech Synthesiscited by 35
  11. 2022iSTFTNet: Fast and Lightweight Mel-Spectrogram Vocoder Incorporating Inverse Short-Time Fourier Transformcited by 30
  12. 2022Towards Building Text-To-Speech Systems for the Next Billion Userscited by 27
38 more
  1. 2023AI-Generated Content (AIGC) for Various Data Modalities: A Surveycited by 25
  2. 2021ESPnet2-TTS: Extending the Edge of TTS Researchcited by 22
  3. 2022WaveFit: An Iterative and Non-autoregressive Neural Vocoder based on Fixed-Point Iterationcited by 20
  4. 2024SafeEar: Content Privacy-Preserving Audio Deepfake Detectioncited by 20
  5. 2022Avocodo: Generative Adversarial Network for Artifact-free Vocodercited by 18
  6. 2021AnyoneNet: Synchronized Speech and Talking Head Generation for Arbitrary Personcited by 17
  7. 2021GAN Vocoder: Multi-Resolution Discriminator Is All You Needcited by 16
  8. 2021HUI-Audio-Corpus-German: A high quality TTS datasetcited by 14
  9. 2022An Initial Investigation for Detecting Vocoder Fingerprints of Fake Audiocited by 12
  10. 2022Modelling black-box audio effects with time-varying feature modulationcited by 11
  11. 2024Where are we in audio deepfake detection? A systematic analysis over generative and detection modelscited by 11
  12. 2022SpecRNet: Towards Faster and More Accessible Audio DeepFake Detectioncited by 9
  13. 2024RFWave: Multi-band Rectified Flow for Audio Waveform Reconstructioncited by 9
  14. 2021VocBench: A Neural Vocoder Benchmark for Speech Synthesiscited by 8
  15. 2022Lightweight and High-Fidelity End-to-End Text-to-Speech with Multi-Band Generation and Inverse Short-Time Fourier Transformcited by 8
  16. 2024RapVerse: Coherent Vocals and Whole-Body Motions Generations from Textcited by 7
  17. 2021Review of end-to-end speech synthesis technology based on deep learningcited by 6
  18. 2021A Streamwise GAN Vocoder for Wideband Speech Coding at Very Low Bit Ratecited by 5
  19. 2023Wave-U-Net Discriminator: Fast and Lightweight Discriminator for Generative Adversarial Network-Based Speech Synthesiscited by 4
  20. 2021Unified Source-Filter GAN: Unified Source-filter Network Based On Factorization of Quasi-Periodic Parallel WaveGANcited by 3
  21. 2023Continuous descriptor-based control for deep audio synthesiscited by 3
  22. 2023Large-scale unsupervised audio pre-training for video-to-speech synthesiscited by 3
  23. 2023iSTFTNet2: Faster and More Lightweight iSTFT-Based Neural Vocoder Using 1D-2D CNNcited by 3
  24. 2024Lightweight Model Attribution and Detection of Synthetic Speech via Audio Residual Fingerprintscited by 3
  25. 2021Improving Prosody for Unseen Texts in Speech Synthesis by Utilizing Linguistic Information and Noisy Datacited by 2
  26. 2022A Study of Modeling Rising Intonation in Cantonese Neural Speech Synthesiscited by 2
  27. 2023Towards generalizing deep-audio fake detection networkscited by 2
  28. 2024Ultra-lightweight Neural Differential DSP Vocoder For High Quality Speech Synthesiscited by 2
  29. 2025DAFMSVC: One-Shot Singing Voice Conversion with Dual Attention Mechanism and Flow Matchingcited by 2
  30. 2025Vocoder-Projected Feature Discriminatorcited by 2
  31. 2021Improve GAN-based Neural Vocoder using Pointwise Relativistic LeastSquare GANcited by 1
  32. 2022LibriS2S: A German-English Speech-to-Speech Translation Corpuscited by 1
  33. 2022End-to-end speech recognition modeling from de-identified datacited by 1
  34. 2024EVA-GAN: Enhanced Various Audio Generation via Scalable Generative Adversarial Networkscited by 1
  35. 2025Lightweight End-to-end Text-to-speech Synthesis for low resource on-device applicationscited by 1
  36. 2025Quantize More, Lose Less: Autoregressive Generation from Residually Quantized Speech Representationscited by 1
  37. 2026VidAudio-Bench: Benchmarking V2A and VT2A Generation across Four Audio Categoriescited by 1
  38. 2021Sprachsynthese -- State-of-the-Art in englischer und deutscher Sprachecited by 0

50 most cited of 63 All 63 (JSON)