# Lightweight End-to-end Text-to-speech Synthesis for low resource on-device applications

arXiv: https://arxiv.org/abs/2505.07701 · HTML: https://mlsanity.com/paper/2505.07701/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 32 references that are in the map.
- First posted on arXiv 12 May 2025.

## Builds on (most cited first, 27 of 27)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis](https://mlsanity.com/paper/2010.05646/) (2020)
- [Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions](https://mlsanity.com/paper/1712.05884/) (2017)
- [FastSpeech 2: Fast and High-Quality End-to-End Text to Speech](https://mlsanity.com/paper/2006.04558/) (2020)
- [Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech](https://mlsanity.com/paper/2106.06103/) (2021)
- [MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis](https://mlsanity.com/paper/1910.06711/) (2019)
- [Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram](https://mlsanity.com/paper/1910.11480/) (2019)
- [Glow-TTS: A Generative Flow for Text-to-Speech via Monotonic Alignment Search](https://mlsanity.com/paper/2005.11129/) (2020)
- [BigVGAN: A Universal Neural Vocoder with Large-Scale Training](https://mlsanity.com/paper/2206.04658/) (2022)
- [A Survey on Neural Speech Synthesis](https://mlsanity.com/paper/2106.15561/) (2021)
- [NaturalSpeech: End-to-End Text to Speech Synthesis with Human-Level Quality](https://mlsanity.com/paper/2205.04421/) (2022)
- [Multi-band MelGAN: Faster Waveform Generation for High-Quality Text-to-Speech](https://mlsanity.com/paper/2005.05106/) (2020)
- [Wave-Tacotron: Spectrogram-free end-to-end text-to-speech synthesis](https://mlsanity.com/paper/2011.03568/) (2020)
- [F0-consistent many-to-many non-parallel voice conversion via conditional autoencoder](https://mlsanity.com/paper/2004.07370/) (2020)
- [DurIAN: Duration Informed Attention Network For Multimodal Synthesis](https://mlsanity.com/paper/1909.01700/) (2019)
- [LightSpeech: Lightweight and Fast Text to Speech with Neural Architecture Search](https://mlsanity.com/paper/2102.04040/) (2021)
- [JETS: Jointly Training FastSpeech2 and HiFi-GAN for End to End Text to Speech](https://mlsanity.com/paper/2203.16852/) (2022)
- [ESPnet2-TTS: Extending the Edge of TTS Research](https://mlsanity.com/paper/2110.07840/) (2021)
- [SpeedySpeech: Efficient Neural Speech Synthesis](https://mlsanity.com/paper/2008.03802/) (2020)
- [On-device neural speech synthesis](https://mlsanity.com/paper/2109.08710/) (2021)
- [Lightweight and High-Fidelity End-to-End Text-to-Speech with Multi-Band Generation and Inverse Short-Time Fourier Transform](https://mlsanity.com/paper/2210.15975/) (2022)
- [Low-data? No problem: low-resource, language-agnostic conversational text-to-speech via F0-conditioned data augmentation](https://mlsanity.com/paper/2207.14607/) (2022)
- [AdaVocoder: Adaptive Vocoder for Custom Voice](https://mlsanity.com/paper/2203.09825/) (2022)
- [A Cyclical Post-filtering Approach to Mismatch Refinement of Neural Vocoder for Text-to-speech Systems](https://mlsanity.com/paper/2005.08659/) (2020)

## Built on by (most cited first, 1 of 1)

- [Unseen Speaker and Language Adaptation for Lightweight Text-To-Speech with Adapters](https://mlsanity.com/paper/2508.18006/) (2025)

## Abstract (arXiv)

Recent works have shown that modelling raw waveform directly from text in an end-to-end (E2E) fashion produces more natural-sounding speech than traditional neural text-to-speech (TTS) systems based on a cascade or two-stage approach. However, current E2E state-of-the-art models are computationally complex and memory-consuming, making them unsuitable for real-time offline on-device applications in low-resource scenarios. To address this issue, we propose a Lightweight E2E-TTS (LE2E) model that generates high-quality speech requiring minimal computational resources. We evaluate the proposed model on the LJSpeech dataset and show that it achieves state-of-the-art performance while being up to 90% smaller in terms of model parameters and 10× faster in real-time-factor. Furthermore, we demonstrate that the proposed E2E training paradigm achieves better quality compared to an equivalent architecture trained in a two-stage approach. Our results suggest that LE2E is a promising approach for developing real-time, high quality, low-resource TTS applications for on-device applications.