mlsanityData to 7 Oct 2026

JETS: Jointly Training FastSpeech2 and HiFi-GAN for End to End Text to Speech

arXiv 2203.16852 ↗First posted 31 Mar 2022eess.AS cs.LG cs.SDcited by 26

Not on the map. Why?

Abstract (arXiv)

In neural text-to-speech (TTS), two-stage system or a cascade of separately learned models have shown synthesis quality close to human speech. For example, FastSpeech2 transforms an input text to a mel-spectrogram and then HiFi-GAN generates a raw waveform from a mel-spectogram where they are called an acoustic feature generator and a neural vocoder respectively. However, their training pipeline is somewhat cumbersome in that it requires a fine-tuning and an accurate speech-text alignment for optimal performance. In this work, we present end-to-end text-to-speech (E2E-TTS) model which has a simplified training pipeline and outperforms a cascade of separately learned models. Specifically, our proposed model is jointly trained FastSpeech2 and HiFi-GAN with an alignment module. Since there is no acoustic feature mismatch between training and inference, it does not requires fine-tuning. Furthermore, we remove dependency on an external speech-text alignment tool by adopting an alignment learning objective in our joint training framework. Experiments on LJSpeech corpus shows that the proposed model outperforms publicly available, state-of-the-art implementations of ESPNet2-TTS on subjective evaluation (MOS) and some objective evaluations.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Builds on 21 of its 27 references that are in the map. First posted on arXiv 31 Mar 2022.

Builds on

21 of 27 references.

References · 21

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  3. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  4. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  5. 2016Least Squares Generative Adversarial Networkscited by 1,000
  6. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
  7. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
  8. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 555
  9. 2019MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesiscited by 351
  10. 2019Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogramcited by 337
  11. 2018WaveGlow: A Flow-based Generative Network for Speech Synthesiscited by 308
  12. 2020Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Searchcited by 269
9 more
  1. 2018Efficient Neural Audio Synthesiscited by 218
  2. 2020FastPitch: Parallel Text-to-speech with Pitch Predictioncited by 150
  3. 2020End-to-End Adversarial Text-to-Speechcited by 77
  4. 2021UnivNet: A Neural Vocoder with Multi-Resolution Spectrogram Discriminators for High-Fidelity Waveform Generationcited by 71
  5. 2021One TTS Alignment To Rule Them Allcited by 47
  6. 2020Wave-Tacotron: Spectrogram-free end-to-end text-to-speech synthesiscited by 38
  7. 2021ESPnet2-TTS: Extending the Edge of TTS Researchcited by 22
  8. 2020EfficientTTS: An Efficient and High-Quality Text-to-Speech Architecturecited by 14
  9. 2020JDI-T: Jointly trained Duration Informed Transformer for Text-To-Speech without Explicit Alignmentcited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 10; 2024: 5; 2025: 3; 2026: 4.

26 papers.

Sort:

By area:

  1. 2024NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Modelscited by 222
  2. 2023StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Modelscited by 148
  3. 2022StyleTTS: A Style-Based Generative Model for Natural and Diverse Text-to-Speech Synthesiscited by 32
  4. 2023VITS2: Improving Quality and Efficiency of Single-Stage Text-to-Speech with Adversarial Learning and Architecture Designcited by 25
  5. 2024PeriodWave: Multi-Period Flow Matching for High-Fidelity Waveform Generationcited by 12
  6. 2022Source-Filter HiFi-GAN: Fast and Pitch Controllable High-Fidelity Neural Vocodercited by 11
  7. 2022Unsupervised TTS Acoustic Modeling for TTS with Conditional Disentangled Sequential VAEcited by 7
  8. 2022Period VITS: Variational Inference with Explicit Pitch Modeling for End-to-end Emotional Speech Synthesiscited by 6
  9. 2024Noise-robust zero-shot text-to-speech synthesis conditioned on self-supervised speech-representation model with adapterscited by 4
  10. 2023Towards Robust FastSpeech 2 by Modelling Residual Multimodalitycited by 2
  11. 2023Rapid Speaker Adaptation in Low Resource Text to Speech Systems using Synthetic Data and Transfer learningcited by 2
  12. 2025Continuous Autoregressive Modeling with Stochastic Monotonic Alignment for Speech Synthesiscited by 2
14 more
  1. 2025BitTTS: Highly Compact Text-to-Speech Using 1.58-bit Quantization and Weight Indexingcited by 2
  2. 2026Deepfake Word Detection by Next-token Prediction using Fine-tuned Whispercited by 2
  3. 2023DC CoMix TTS: An End-to-End Expressive TTS with Discrete Code Collaborated with Mixercited by 1
  4. 2025Lightweight End-to-end Text-to-speech Synthesis for low resource on-device applicationscited by 1
  5. 2026WavTTS: Towards High-Quality Zero-Shot TTS via Direct Raw Waveform Modelingcited by 1
  6. 2023Singing voice synthesis based on frame-level sequence-to-sequence models considering vocal timing deviationcited by 0
  7. 2023SALTTS: Leveraging Self-Supervised Speech Representations for improved Text-to-Speech Synthesiscited by 0
  8. 2023Generative Adversarial Training for Text-to-Speech Synthesis Based on Raw Phonetic Input and Explicit Prosody Modellingcited by 0
  9. 2023Compression of end-to-end non-autoregressive image-to-speech system for low-resourced devicescited by 0
  10. 2023Code-Mixed Text to Speech Synthesis under Low-Resource Constraintscited by 0
  11. 2024PRESENT: Zero-Shot Text-to-Prosody Controlcited by 0
  12. 2024Training Universal Vocoders with Feature Smoothing-Based Augmentation Methods for High-Quality TTS Systemscited by 0
  13. 2026ZeSTA: Zero-Shot TTS Augmentation with Domain-Conditioned Training for Data-Efficient Personalized Speech Synthesiscited by 0
  14. 2026MambaVoiceCloning: Efficient and Expressive Text-to-Speech via State-Space Modeling and Diffusion Controlcited by 0