VoXtream2: Full-stream TTS with dynamic speaking rate control
Not on the map. Why?
Abstract (arXiv)
Full-stream text-to-speech (TTS) for interactive systems must start speaking with minimal delay while remaining controllable as text arrives incrementally. We present VoXtream2, a zero-shot full-stream TTS model with dynamic speaking-rate control that can be updated mid-utterance on the fly. VoXtream2 combines a distribution matching mechanism over duration states with classifier-free guidance across conditioning signals to improve controllability and synthesis quality. Prompt-text masking enables textless audio prompting, removing the need for prompt transcription. Across standard zero-shot benchmarks and a dedicated speaking-rate test set, VoXtream2 achieves competitive objective and subjective results against public baselines despite a smaller model and less training data. In full-stream mode, it runs 4 times faster than real time with 74 ms first-packet latency on a consumer GPU.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 50 of its 65 references that are in the map. First posted on arXiv 13 Mar 2026.
Builds on
50 of 65 references.
References · 50
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2022Classifier-Free Diffusion Guidancecited by 5,847
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
- 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
- 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
- 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 752
- 2020ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verificationcited by 628
- 2024Moshi: a speech-text foundation model for real-time dialoguecited by 568
- 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 390
- 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scalecited by 379
- 2020GeDi: Generative Discriminator Guided Sequence Generationcited by 378
- 2021FUDGE: Controlled Text Generation With Future Discriminatorscited by 376
38 more
- 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matchingcited by 361
- 2022AudioGen: Textually Guided Audio Generationcited by 295
- 2024CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Modelscited by 288
- 2024NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Modelscited by 226
- 2024Emilia: An Extensive, Multilingual, and Diverse Speech Dataset for Large-Scale Speech Generationcited by 204
- 2024XTTS: a Massively Multilingual Zero-Shot Text-to-Speech Modelcited by 179
- 2024MaskGCT: Zero-Shot Text-to-Speech with Masked Generative Codec Transformercited by 161
- 2023Matcha-TTS: A fast TTS architecture with conditional flow matchingcited by 145
- 2022Exact Feature Distribution Matching for Arbitrary Style Transfer and Domain Generalizationcited by 122
- 2025Spark-TTS: An Efficient LLM-Based Text-to-Speech Model with Single-Stream Decoupled Speech Tokenscited by 113
- 2022PromptTTS: Controllable Text-to-Speech with Text Descriptionscited by 110
- 2023InstructTTS: Modelling Expressive TTS in Discrete Latent Space with Natural Language Style Promptcited by 92
- 2025IndexTTS2: A Breakthrough in Emotionally Expressive and Duration-Controlled Auto-Regressive Zero-Shot Text-to-Speechcited by 90
- 2024Natural language guidance of high-fidelity text-to-speech with synthetic annotationscited by 67
- 2023LibriSpeech-PC: Benchmark for Evaluation of Punctuation and Capitalization Capabilities of end-to-end ASR Modelscited by 49
- 2023PromptTTS 2: Describing and Generating Voices with Text Promptcited by 48
- 2023PromptStyle: Controllable Style Transfer for Text-to-Speech with Natural Language Descriptionscited by 34
- 2025ZipVoice: Fast and High-Quality Zero-Shot Text-to-Speech with Flow Matchingcited by 30
- 2024Reshape Dimensions Network for Speaker Recognitioncited by 28
- 2025Koel-TTS: Enhancing LLM based Speech Generation with Preference Alignment and Classifier Free Guidancecited by 27
- 2025Streaming Sequence-to-Sequence Learning with Delayed Streams Modelingcited by 27
- 2023PromptTTS++: Controlling Speaker Identity in Prompt-Based Text-to-Speech Using Natural Language Descriptionscited by 25
- 2024ControlSpeech: Towards Simultaneous and Independent Zero-shot Speaker Cloning and Zero-shot Language Style Controlcited by 17
- 2023The taste of IPA: Towards open-vocabulary keyword spotting and forced alignment in any languagecited by 14
- 2024SSR-Speech: Towards Stable, Safe and Robust Zero-shot Text-based Speech Editing and Synthesiscited by 13
- 2025HiFiTTS-2: A Large-Scale High Bandwidth Speech Datasetcited by 10
- 2026VoiceSculptor: Your Voice, Designed By Youcited by 10
- 2020Speaking Speed Control of End-to-End Speech Synthesis using Sentence-Level Conditioningcited by 8
- 2025SpeakStream: Streaming Text-to-Speech with Interleaved Datacited by 8
- 2025Sidon: Fast and Robust Open-Source Multilingual Speech Restoration for Large-scale Dataset Cleansingcited by 8
- 2025LLMVoX: Autoregressive Streaming Text-to-Speech Model for Any LLMcited by 7
- 2025Word-Level Emotional Expression Control in Zero-Shot Text-to-Speech Synthesiscited by 7
- 2024Interleaved Speech-Text Language Models for Simple Streaming Text-to-Speech Synthesiscited by 6
- 2025Cross-Lingual F5-TTS: Towards Language-Agnostic Voice Cloning and Speech Synthesiscited by 5
- 2025VoXtream: Full-Stream Text-to-Speech with Extremely Low Latencycited by 5
- 2025VoiceStar: Robust Zero-Shot Autoregressive TTS with Duration Control and Extrapolationcited by 4
- 2023Speaking rate attention-based duration prediction for speed control TTScited by 2
- 2026Zero-Shot TTS With Enhanced Audio Prompts: Bsc Submission For The 2026 Wildspoof Challenge TTS Trackcited by 2
Built on by
Citing papers per year: 2026: 3.
3 papers.