mlsanityData to 7 Oct 2026

On-device neural speech synthesis

arXiv 2109.08710 ↗First posted 17 Sep 2021eess.AS cs.CL cs.PFcited by 9

Not on the map. Why?

Abstract (arXiv)

Recent advances in text-to-speech (TTS) synthesis, such as Tacotron and WaveRNN, have made it possible to construct a fully neural network based TTS system, by coupling the two components together. Such a system is conceptually simple as it only takes grapheme or phoneme input, uses Mel-spectrogram as an intermediate feature, and directly generates speech samples. The system achieves quality equal or close to natural speech. However, the high computational cost of the system and issues with robustness have limited their usage in real-world speech synthesis applications and products. In this paper, we present key modeling improvements and optimization strategies that enable deploying these models, not only on GPU servers, but also on mobile devices. The proposed system can generate high-quality 24 kHz speech at 5x faster than real time on server and 3x faster than real time on mobile devices.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 21 of its 25 references that are in the map. First posted on arXiv 17 Sep 2021.

Builds on

21 of 25 references.

References · 21

  1. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  2. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  3. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,135
  4. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
  5. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
  6. 2017Tacotron: Towards End-to-End Speech Synthesiscited by 487
  7. 2019MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesiscited by 351
  8. 2018WaveGlow: A Flow-based Generative Network for Speech Synthesiscited by 308
  9. 2018Efficient Neural Audio Synthesiscited by 218
  10. 2017Parallel WaveNet: Fast High-Fidelity Speech Synthesiscited by 214
  11. 2016SampleRNN: An Unconditional End-to-End Neural Audio Generation Modelcited by 110
  12. 2018LPCNet: Improving Neural Speech Synthesis Through Linear Predictioncited by 109
9 more
  1. 2017Online and Linear-Time Attention by Enforcing Monotonic Alignmentscited by 85
  2. 2017Deep Voice: Real-time Neural Text-to-Speechcited by 83
  3. 2020End-to-End Adversarial Text-to-Speechcited by 77
  4. 2017Deep Voice 2: Multi-Speaker Neural Text-to-Speechcited by 75
  5. 2018ClariNet: Parallel Wave Generation in End-to-End Text-to-Speechcited by 70
  6. 2017Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learningcited by 67
  7. 2020Parallel Tacotron: Non-Autoregressive and Controllable TTScited by 49
  8. 2020Wave-Tacotron: Spectrogram-free end-to-end text-to-speech synthesiscited by 38
  9. 2019Robust Sequence-to-Sequence Acoustic Modeling with Stepwise Monotonic Attention for Neural TTScited by 33

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 1; 2023: 3; 2024: 0; 2025: 2; 2026: 1.

9 papers.

  1. 2021Hierarchical prosody modeling and control in non-autoregressive parallel neural TTScited by 9
  2. 2021Emphasis control for parallel neural TTScited by 9
  3. 2022Space-Efficient Representation of Entity-centric Query Language Modelscited by 7
  4. 2023EfficientSpeech: An On-Device Text to Speech Modelcited by 4
  5. 2023Server-side Rescoring of Spoken Entity-centric Knowledge Queries for Virtual Assistantscited by 2
  6. 2025Compact Neural TTS Voices for Accessibilitycited by 2
  7. 2025Lightweight End-to-end Text-to-speech Synthesis for low resource on-device applicationscited by 1
  8. 2023DTW-SiameseNet: Dynamic Time Warped Siamese Network for Mispronunciation Detection and Correctioncited by 0
  9. 2026Same Bit Width, Different Outcomes: Post-Training Quantization of Text-to-Speech Across Architecturescited by 0