mlsanityData to 7 Oct 2026

ESPnet2-TTS: Extending the Edge of TTS Research

arXiv 2110.07840 ↗First posted 15 Oct 2021cs.CL cs.SD eess.AScited by 22

Not on the map. Why?

Abstract (arXiv)

This paper describes ESPnet2-TTS, an end-to-end text-to-speech (E2E-TTS) toolkit. ESPnet2-TTS extends our earlier version, ESPnet-TTS, by adding many new features, including: on-the-fly flexible pre-processing, joint training with neural vocoders, and state-of-the-art TTS models with extensions like full-band E2E text-to-waveform modeling, which simplify the training pipeline and further enhance TTS performance. The unified design of our recipes enables users to quickly reproduce state-of-the-art E2E-TTS results. We also provide many pre-trained models in a unified Python interface for inference, offering a quick means for users to generate baseline samples and build demos. Experimental evaluations with English and Japanese corpora demonstrate that our provided models synthesize utterances comparable to ground-truth ones, achieving state-of-the-art TTS performance. The toolkit is available online at https://github.com/espnet/espnet.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 32 of its 51 references that are in the map. First posted on arXiv 15 Oct 2021.

Builds on

32 of 51 references.

References · 32

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  3. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  4. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  5. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
  6. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
  7. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 555
  8. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 554
  9. 2017Tacotron: Towards End-to-End Speech Synthesiscited by 487
  10. 2019MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesiscited by 351
  11. 2019Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogramcited by 337
  12. 2018Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesiscited by 280
20 more
  1. 2018Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesiscited by 257
  2. 2019NeMo: a toolkit for building AI applications using Neural Modulescited by 230
  3. 2020FastPitch: Parallel Text-to-speech with Pitch Predictioncited by 150
  4. 2020Recent Developments on ESPnet Toolkit Boosted by Conformercited by 132
  5. 2019High Fidelity Speech Synthesis with Adversarial Networkscited by 90
  6. 2020Voice Conversion Challenge 2020: Intra-lingual semi-parallel and cross-lingual voice conversioncited by 75
  7. 2021WaveGrad 2: Iterative Refinement for Text-to-Speech Synthesiscited by 65
  8. 2020Multi-band MelGAN: Faster Waveform Generation for High-Quality Text-to-Speechcited by 63
  9. 2019ESPnet-TTS: Unified, Reproducible, and Integratable Open Source End-to-End Text-to-Speech Toolkitcited by 60
  10. 2017Efficiently Trainable Text-to-Speech System Based on Deep Convolutional Networks with Guided Attentioncited by 57
  11. 2017JSUT corpus: free large-scale Japanese speech corpus for end-to-end speech synthesiscited by 57
  12. 2020Asteroid: the PyTorch-based audio source separation toolkit for researcherscited by 45
  13. 2020MultiSpeech: Multi-Speaker Text to Speech with Transformercited by 42
  14. 2020Fully-hierarchical fine-grained prosody modeling for interpretable speech synthesiscited by 41
  15. 2019Voice Transformer Network: Sequence-to-Sequence Voice Conversion Using Transformer with Text-to-Speech Pretrainingcited by 29
  16. 2020StyleMelGAN: An Efficient High-Fidelity Adversarial Vocoder with Temporal Adaptive Normalizationcited by 26
  17. 2021fairseq S^2: A Scalable and Integrable Speech Synthesis Toolkitcited by 24
  18. 2019Chainer: A Deep Learning Framework for Accelerating the Research Cyclecited by 22
  19. 2019JVS corpus: free Japanese multi-speaker voice corpuscited by 20
  20. 2018Neural Speech Synthesis with Transformer Networkcited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 8; 2023: 6; 2024: 4; 2025: 1; 2026: 3.

22 papers.

Sort:

By area:

  1. 2022StyleTTS: A Style-Based Generative Model for Natural and Diverse Text-to-Speech Synthesiscited by 32
  2. 2022JETS: Jointly Training FastSpeech2 and HiFi-GAN for End to End Text to Speechcited by 26
  3. 2023A Vector Quantized Approach for Text to Speech Synthesis on Real-World Spontaneous Speechcited by 22
  4. 2024ESPnet-SPK: full pipeline speaker embedding toolkit with reproducible recipes, self-supervised front-ends, and off-the-shelf modelscited by 22
  5. 2022Language-Agnostic Meta-Learning for Low-Resource Text-to-Speech with Articulatory Featurescited by 20
  6. 2024Exploring the Capability of Mamba in Speech Applicationscited by 19
  7. 2022Low-Resource Multilingual and Zero-Shot Multispeaker TTScited by 15
  8. 2022Exact Prosody Cloning in Zero-Shot Multispeaker Text-to-Speechcited by 13
  9. 2023DiffVoice: Text-to-Speech with Latent Diffusioncited by 13
  10. 2022Structured State Space Decoder for Speech Recognition and Synthesiscited by 11
  11. 2023Learning to Speak from Text: Zero-Shot Multilingual Text-to-Speech with Unsupervised Text Pretrainingcited by 10
  12. 2023ESPnet-ST-v2: Multipurpose Spoken Language Translation Toolkitcited by 10
10 more
  1. 2022Speaker Anonymization with Phonetic Intermediate Representationscited by 9
  2. 2023Improving Cascaded Unsupervised Speech Translation with Denoising Back-translationcited by 5
  3. 2024Llama-VITS: Enhancing TTS Synthesis with Semantic Awarenesscited by 1
  4. 2025Lightweight End-to-end Text-to-speech Synthesis for low resource on-device applicationscited by 1
  5. 2026Do Neural Codecs Generalize? A Controlled Study Across Unseen Languages and Non-Speech Taskscited by 1
  6. 2022Empirical Study Incorporating Linguistic Knowledge on Filled Pauses for Personalized Spontaneous Speech Synthesiscited by 0
  7. 2023Enhancing Speech-to-Speech Translation with Multiple TTS Targetscited by 0
  8. 2024ESPnet-EZ: Python-only ESPnet for Easy Fine-tuning and Integrationcited by 0
  9. 2026Voice Mapping of Text-to-Speech Systems: A Metric-Based Approach for Voice Quality Assessmentcited by 0
  10. 2026TRILOGUE: A Trilingual Spoken Dialogue Fact-Checking Benchmark with Evidence and Paired Audiocited by 0