Lightweight End-to-end Text-to-speech Synthesis for low resource on-device applications
Not on the map. Why?
Abstract (arXiv)
Recent works have shown that modelling raw waveform directly from text in an end-to-end (E2E) fashion produces more natural-sounding speech than traditional neural text-to-speech (TTS) systems based on a cascade or two-stage approach. However, current E2E state-of-the-art models are computationally complex and memory-consuming, making them unsuitable for real-time offline on-device applications in low-resource scenarios. To address this issue, we propose a Lightweight E2E-TTS (LE2E) model that generates high-quality speech requiring minimal computational resources. We evaluate the proposed model on the LJSpeech dataset and show that it achieves state-of-the-art performance while being up to 90% smaller in terms of model parameters and 10× faster in real-time-factor. Furthermore, we demonstrate that the proposed E2E training paradigm achieves better quality compared to an equivalent architecture trained in a two-stage approach. Our results suggest that LE2E is a promising approach for developing real-time, high quality, low-resource TTS applications for on-device applications.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 27 of its 32 references that are in the map. First posted on arXiv 12 May 2025.
Builds on
27 of 32 references.
References · 27
- 2014Adam: A Method for Stochastic Optimizationcited by 37,240
- 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
- 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
- 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,488
- 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,154
- 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 804
- 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 717
- 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 561
- 2019MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesiscited by 354
- 2019Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogramcited by 340
- 2020Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Searchcited by 273
- 2022BigVGAN: A Universal Neural Vocoder with Large-Scale Trainingcited by 251
15 more
- 2021A Survey on Neural Speech Synthesiscited by 184
- 2022NaturalSpeech: End-to-End Text to Speech Synthesis with Human-Level Qualitycited by 152
- 2020Multi-band MelGAN: Faster Waveform Generation for High-Quality Text-to-Speechcited by 63
- 2020Wave-Tacotron: Spectrogram-free end-to-end text-to-speech synthesiscited by 38
- 2020F0-consistent many-to-many non-parallel voice conversion via conditional autoencodercited by 35
- 2019DurIAN: Duration Informed Attention Network For Multimodal Synthesiscited by 33
- 2021LightSpeech: Lightweight and Fast Text to Speech with Neural Architecture Searchcited by 28
- 2022JETS: Jointly Training FastSpeech2 and HiFi-GAN for End to End Text to Speechcited by 26
- 2021ESPnet2-TTS: Extending the Edge of TTS Researchcited by 22
- 2020SpeedySpeech: Efficient Neural Speech Synthesiscited by 15
- 2021On-device neural speech synthesiscited by 9
- 2022Lightweight and High-Fidelity End-to-End Text-to-Speech with Multi-Band Generation and Inverse Short-Time Fourier Transformcited by 9
- 2022Low-data? No problem: low-resource, language-agnostic conversational text-to-speech via F0-conditioned data augmentationcited by 4
- 2022AdaVocoder: Adaptive Vocoder for Custom Voicecited by 2
- 2020A Cyclical Post-filtering Approach to Mismatch Refinement of Neural Vocoder for Text-to-speech Systemscited by 1
Built on by
Citing papers per year: 2025: 1.
1 paper.