mlsanityData to 14 Oct 2026

EfficientSpeech: An On-Device Text to Speech Model

arXiv 2305.13905 ↗First posted 23 May 2023eess.AS cs.CL cs.SDcited by 4

Not on the map. Why?

Abstract (arXiv)

State of the art (SOTA) neural text to speech (TTS) models can generate natural-sounding synthetic voices. These models are characterized by large memory footprints and substantial number of operations due to the long-standing focus on speech quality with cloud inference in mind. Neural TTS models are generally not designed to perform standalone speech syntheses on resource-constrained and no Internet access edge devices. In this work, an efficient neural TTS called EfficientSpeech that synthesizes speech on an ARM CPU in real-time is proposed. EfficientSpeech uses a shallow non-autoregressive pyramid-structure transformer forming a U-Network. EfficientSpeech has 266k parameters and consumes 90 MFLOPS only or about 1% of the size and amount of computation in modern compact models such as Mixer-TTS. EfficientSpeech achieves an average mel generation real-time factor of 104.3 on an RPi4. Human evaluation shows only a slight degradation in audio quality as compared to FastSpeech2.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 16 of its 28 references that are in the map. First posted on arXiv 23 May 2023.

Builds on

16 of 28 references.

References · 16

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2016Layer Normalizationcited by 5,669
  4. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  5. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,940
  6. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,154
  7. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 804
  8. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 717
  9. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 561
  10. 2019NeMo: a toolkit for building AI applications using Neural Modulescited by 235
  11. 2020FastPitch: Parallel Text-to-speech with Pitch Predictioncited by 151
  12. 2021The Efficiency Misnomercited by 98
4 more
  1. 2021PortaSpeech: Portable and High-Quality Generative Text-to-Speechcited by 47
  2. 2021LightSpeech: Lightweight and Fast Text to Speech with Neural Architecture Searchcited by 28
  3. 2021On-device neural speech synthesiscited by 9
  4. 2021Mixer-TTS: non-autoregressive, fast and compact text-to-speech model conditioned on language model embeddingscited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 1; 2025: 1; 2026: 1.

4 papers.

  1. 2023DCTTS: Discrete Diffusion Model with Contrastive Learning for Text-to-speech Generationcited by 9
  2. 2024RU-AI: A Large Multimodal Dataset for Machine-Generated Content Detectioncited by 4
  3. 2025Edge-Based Speech Transcription and Synthesis for Kinyarwanda and Swahili Languagescited by 1
  4. 2026GrainSpeech: Less Context, More Detail for Compact Speech Synthesiscited by 0