mlsanityData to 19 Oct 2026

StellarTTS: Sparse Temporal Embedding for Low-Latency and Robust Speech Synthesis

arXiv 2607.19859 ↗First posted 22 Jul 2026cs.SDcited by 0

On the map: Speech and audio language models → Vocoders and speech synthesis

Abstract (arXiv)

The trade-off between robustness, latency, and prosody critically challenges text-to-speech (TTS) systems. Autoregressive models, despite fidelity, are slow and error-prone; non-autoregressive (NAR) alternatives, while fast, often sacrifice prosodic naturalness via rigid alignments. This paper introduces StellarTTS, a novel mobile-optimized NAR TTS framework based on a sparse temporal embedding strategy, enabling granular control of phoneme duration, pronunciation, and prosody. Furthermore, we propose a semantic-aware codec that facilitates efficient single-stage decoding. Conditioned on the sparse temporal embedding, our 83M-parameter lightweight masked generative transformer achieves a real-time factor (RTF) of 0.08. Experiments demonstrate that StellarTTS attains lower latency and stronger robustness compared to state-of-the-art TTS systems, while maintaining competitive performance in audio quality, prosodic naturalness, and speaker similarity.

Key facts. No citing papers in the map yet (data through 19 Oct 2026). Placed in Vocoders and speech synthesis, one of 11 directions in Speech and audio language models. Builds on 22 of its 23 references that are in the map. First posted on arXiv 22 Jul 2026.

Builds on

22 of 23 references.

Closest ancestors · 12

Same direction, newest first

  1. 2024CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Models13 Dec
  2. 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching9 Oct
  3. 2024FireRedTTS: A Foundation Text-To-Speech Framework for Industry-Level Generative Speech Applications5 Sep
  4. 2024MaskGCT: Zero-Shot Text-to-Speech with Masked Generative Codec Transformer1 Sep
  5. 2024CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokens7 Jul
  6. 2024E2 TTS: Embarrassingly Easy Fully Non-Autoregressive Zero-Shot TTS26 Jun
  7. 2024Seed-TTS: A Family of High-Quality Versatile Speech Generation Models4 Jun
  8. 2024NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Models5 Mar
  9. 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scale23 Jun
  10. 2023NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizers18 Apr
  11. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers5 Jan
  12. 2021SoundStream: An End-to-End Neural Audio Codec7 Jul

Other references · 10

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,599
  2. 2017Neural Discrete Representation Learningcited by 4,843
  3. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,762
  4. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 2,135
  5. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 947
  6. 2022High Fidelity Neural Audio Compressioncited by 917
  7. 2021W2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Trainingcited by 412
  8. 2024Emilia: An Extensive, Multilingual, and Diverse Speech Dataset for Large-Scale Speech Generationcited by 268
  9. 2023SoundStorm: Efficient Parallel Audio Generationcited by 154
  10. 20233D-Speaker: A Large-Scale Multi-Device, Multi-Distance, and Multi-Dialect Corpus for Speech Representation Disentanglementcited by 31

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026CuteTTS: Efficient and High-Quality Speech Synthesis via Autoregressive Modeling of Continuous Latents
  2. 2022AutoTTS: End-to-End Text-to-Speech Synthesis through Differentiable Duration Modeling
  3. 2025STCTS: Generative Semantic Compression for Ultra-Low Bitrate Speech via Explicit Text-Prosody-Timbre Decomposition
  4. 2025BridgeCode: A Dual Speech Representation Paradigm for Autoregressive Zero-Shot Text-to-Speech Synthesis
  5. 2025CLEAR: Continuous Latent Autoregressive Modeling for High-quality and Low-latency Speech Synthesis
  6. 2021Mixer-TTS: non-autoregressive, fast and compact text-to-speech model conditioned on language model embeddings
  7. 2025SupertonicTTS: Towards Highly Efficient and Streamlined Text-to-Speech System
  8. 2026Combining Masked Language Modeling and Cross-Modal Contrastive Learning for Prosody-Aware TTS