mlsanityData to 7 Oct 2026

GANSpeech: Adversarial Training for High-Fidelity Multi-Speaker Speech Synthesis

arXiv 2106.15153 ↗First posted 29 Jun 2021eess.AS cs.CL cs.LGcited by 18

Not on the map. Why?

Abstract (arXiv)

Recent advances in neural multi-speaker text-to-speech (TTS) models have enabled the generation of reasonably good speech quality with a single model and made it possible to synthesize the speech of a speaker with limited training data. Fine-tuning to the target speaker data with the multi-speaker model can achieve better quality, however, there still exists a gap compared to the real speech sample and the model depends on the speaker. In this work, we propose GANSpeech, which is a high-fidelity multi-speaker TTS model that adopts the adversarial training method to a non-autoregressive multi-speaker TTS model. In addition, we propose simple but efficient automatic scaling methods for feature matching loss used in adversarial training. In the subjective listening tests, GANSpeech significantly outperformed the baseline multi-speaker FastSpeech and FastSpeech2 models, and showed a better MOS score than the speaker-specific fine-tuned FastSpeech2.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 34 of its 38 references that are in the map. First posted on arXiv 29 Jun 2021.

Builds on

34 of 38 references.

References · 34

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2016Improved Techniques for Training GANscited by 2,938
  4. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  5. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,176
  6. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  7. 2016Least Squares Generative Adversarial Networkscited by 1,000
  8. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
  9. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
  10. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 639
  11. 2017Tacotron: Towards End-to-End Speech Synthesiscited by 487
  12. 2015Attention-Based Models for Speech Recognitioncited by 400
22 more
  1. 2019MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesiscited by 351
  2. 2019Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogramcited by 337
  3. 2018Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesiscited by 280
  4. 2017StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 229
  5. 2017Parallel WaveNet: Fast High-Fidelity Speech Synthesiscited by 214
  6. 2020FastPitch: Parallel Text-to-speech with Pitch Predictioncited by 150
  7. 2018Neural Voice Cloning with a Few Samplescited by 100
  8. 2021AdaSpeech: Adaptive Text to Speech for Custom Voicecited by 98
  9. 2017Deep Voice 2: Multi-Speaker Neural Text-to-Speechcited by 75
  10. 2019Zero-Shot Multi-Speaker Text-To-Speech with State-of-the-art Neural Speaker Embeddingscited by 73
  11. 2017Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learningcited by 67
  12. 2018Sample Efficient Adaptive Text-to-Speechcited by 45
  13. 2020MultiSpeech: Multi-Speaker Text to Speech with Transformercited by 42
  14. 2018Semi-Supervised Training for Improving Data Efficiency in End-to-End Speech Synthesiscited by 30
  15. 2020VocGAN: A High-Fidelity Real-time Vocoder with a Hierarchically-nested Adversarial Networkcited by 26
  16. 2019Adversarially Trained End-to-end Korean Singing Voice Synthesis Systemcited by 22
  17. 2020BOFFIN TTS: Few-Shot Speaker Adaptation by Bayesian Optimizationcited by 22
  18. 2020AdaDurIAN: Few-shot Adaptation for Neural Text-to-Speech with DurIANcited by 14
  19. 2019High quality, lightweight and adaptable TTS using LPCNetcited by 11
  20. 2020Speaking Speed Control of End-to-End Speech Synthesis using Sentence-Level Conditioningcited by 8
  21. 2019High-quality Speech Synthesis Using Super-resolution Mel-Spectrogramcited by 7
  22. 2018Speaker-adaptive neural vocoders for parametric speech synthesis systemscited by 1

Built on by

Cited by year

Citing papers per year: 2022: 6; 2023: 5; 2024: 5; 2025: 2.

18 papers.

Sort:

By area:

  1. 2022ProDiff: Progressive Fast Diffusion Model For High-Quality Text-to-Speechcited by 147
  2. 2022DiffGAN-TTS: High-Fidelity and Efficient Text-to-Speech with Denoising Diffusion GANscited by 43
  3. 2024EmoSphere-TTS: Emotional Style and Intensity Modeling via Spherical Emotion Vector for Controllable Emotional Text-to-Speechcited by 23
  4. 2024Towards Controllable Speech Synthesis in the Era of Large Language Models: A Systematic Surveycited by 22
  5. 2023EmoSpeech: Guiding FastSpeech2 Towards Emotional Text to Speechcited by 14
  6. 2023DiffVoice: Text-to-Speech with Latent Diffusioncited by 13
  7. 2022Deep Performer: Score-to-Audio Music Performance Synthesiscited by 12
  8. 2022VarietySound: Timbre-Controllable Video to Sound Generation via Unsupervised Information Disentanglementcited by 11
  9. 2022A Multi-Stage Multi-Codebook VQ-VAE Approach to High-Performance Neural TTScited by 9
  10. 2024DEX-TTS: Diffusion-based EXpressive Text-to-Speech with Style Modeling on Time Variabilitycited by 5
  11. 2025Voice Cloning: Comprehensive Surveycited by 4
  12. 2024Multi-modal Adversarial Training for Zero-Shot Voice Cloningcited by 2
6 more
  1. 2025Emotion Recognition and Generation: A Comprehensive Review of Face, Speech, and Text Modalitiescited by 2
  2. 2022Multi-Task Adversarial Training Algorithm for Multi-Speaker Neural Text-to-Speechcited by 1
  3. 2023FastFit: Towards Real-Time Iterative Neural Vocoder by Replacing U-Net Encoder With Multiple STFTscited by 1
  4. 2023Adversarial Training of Denoising Diffusion Model Using Dual Discriminators for High-Fidelity Multi-Speaker TTScited by 1
  5. 2023Multi-GradSpeech: Towards Diffusion-based Multi-Speaker Text-to-speech Using Consistent Diffusion Modelscited by 1
  6. 2024MultiVerse: Efficient and Expressive Zero-Shot Multi-Task Text-to-Speechcited by 1