mlsanityData to 14 Oct 2026

X2Streaming-TTS: Causal Token-Level Text-to-Speech from Streaming Text with Speech-State Inheritance

arXiv 2608.18661 ↗First posted 19 Aug 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Streaming text-to-speech is essential for low-latency spoken dialogue systems, yet many systems wait for sentence-level text and are therefore only pseudo-streaming. True token-level synthesis must generate speech from uncertain prefixes while maintaining perceptual continuity over an unbounded stream with bounded context. We present X2Streaming-TTS, a causal TTS framework that consumes asynchronously arriving text tokens and emits speech without accessing future input. To handle uncertain prefixes, we introduce causal commitment, which keeps ambiguous expressions provisional through uncertainty-aware buffering and performs capacity-adaptive, punctuation-aware segmentation. To preserve acoustic continuity, we further introduce causal speech-state inheritance, which carries the complete Code2Wav state and selected historical Talker states across segment boundaries. Together with an attention prior constraint, it blocks access to future positions while retaining bounded acoustic context. Experiments show that X2Streaming-TTS outperforms existing pseudo-streaming models on most subjective and objective metrics. Further analysis shows that causal commitment stabilizes online segmentation and reduces failures caused by insufficient context, while speech-state inheritance improves boundary continuity without degrading naturalness or speaker identity. X2Streaming-TTS thus achieves strict token-level synthesis with quality comparable to the evaluated offline baselines, a median time to first audio token (TTFT) of 15.8 ms for a single request, and a median TTFT of 260.8 ms at 128 concurrent requests. Our implementation is publicly available at https://github.com/X-Square-Robot/X2Streaming-TTS .

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 25 of its 34 references that are in the map. First posted on arXiv 19 Aug 2026.

Builds on

25 of 34 references.

References · 25

  1. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  2. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,154
  3. 2021SoundStream: An End-to-End Neural Audio Codeccited by 771
  4. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 752
  5. 2020ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verificationcited by 628
  6. 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 390
  7. 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matchingcited by 361
  8. 2024CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokenscited by 334
  9. 2024Seed-TTS: A Family of High-Quality Versatile Speech Generation Modelscited by 300
  10. 2024StreamingT2V: Consistent, Dynamic, and Extendable Long Video Generation from Textcited by 251
  11. 2023Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesiscited by 153
  12. 2025CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-trainingcited by 150
13 more
  1. 2020Emformer: Efficient Memory Transformer Based Acoustic Model For Low Latency Streaming Speech Recognitioncited by 113
  2. 2022Wespeaker: A Research and Production oriented Speaker Embedding Learning Toolkitcited by 97
  3. 2019Monotonic Multihead Attentioncited by 93
  4. 2024BASE TTS: Lessons from building a billion-parameter Text-to-Speech model on 100K hours of datacited by 75
  5. 2024Segment Any Text: A Universal Approach for Robust, Efficient and Adaptable Sentence Segmentationcited by 41
  6. 2025FireRedTTS-2: Towards Long Conversational Speech Generation for Podcast and Chatbotcited by 35
  7. 2024LiveSpeech: Low-Latency Zero-shot Text-to-Speech via Autoregressive Modeling of Audio Discrete Codescited by 13
  8. 2025SpeakStream: Streaming Text-to-Speech with Interleaved Datacited by 8
  9. 2023Duration-aware pause insertion using pre-trained language model for multi-speaker text-to-speechcited by 5
  10. 2022Automatic Prosody Annotation with Pre-Trained Text-Speech Modelcited by 4
  11. 2025SyncSpeech: Efficient and Low-Latency Text-to-Speech based on Temporal Masked Transformercited by 2
  12. 2026Prosodic Boundary-Aware Streaming Generation for LLM-Based TTS with Streaming Text Inputcited by 1
  13. 2026MagpieTTS-LF: Inference-Time Long-Form Speech Generation Without Training on Long-Form datacited by 1

Built on by

Not cited yet.