CTC-TTS: LLM-based dual-streaming text-to-speech with CTC alignment
Not on the map. Why?
Abstract (arXiv)
Large-language-model (LLM)-based text-to-speech (TTS) systems can generate natural speech, but most are not designed for low-latency dual-streaming synthesis. High-quality dual-streaming TTS depends on accurate text--speech alignment and well-designed training sequences that balance synthesis quality and latency. Prior work often relies on GMM-HMM based forced-alignment toolkits (e.g., MFA), which are pipeline-heavy and less flexible than neural aligners; fixed-ratio interleaving of text and speech tokens struggles to capture text--speech alignment regularities. We propose CTC-TTS, which replaces MFA with a CTC based aligner and introduces a bi-word based interleaving strategy. Two variants are designed: CTC-TTS-L (token concatenation along the sequence length) for higher quality and CTC-TTS-F (embedding stacking along the feature dimension) for lower latency. Experiments show that CTC-TTS outperforms fixed-ratio interleaving and MFA-based baselines on streaming synthesis and zero-shot tasks.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 22 of its 28 references that are in the map. First posted on arXiv 23 Feb 2026.
Builds on
22 of 28 references.
References · 22
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
- 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
- 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
- 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
- 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
- 2021SoundStream: An End-to-End Neural Audio Codeccited by 756
- 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 737
- 2022High Fidelity Neural Audio Compressioncited by 673
- 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 381
- 2024CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokenscited by 323
- 2024Seed-TTS: A Family of High-Quality Versatile Speech Generation Modelscited by 293
- 2024CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Modelscited by 282
10 more
- 2024Mini-Omni: Language Models Can Hear, Talk While Thinking in Streamingcited by 204
- 2024WavTokenizer: an Efficient Acoustic Discrete Codec Tokenizer for Audio Language Modelingcited by 124
- 2024ELLA-V: Stable Neural Codec Language Modeling with Alignment-guided Sequence Reorderingcited by 50
- 2024Whistle: Data-Efficient Multilingual and Crosslingual Speech Recognition via Weakly Phonetic Supervisioncited by 17
- 2024Less Peaky and More Accurate CTC Forced Alignment by Label Priorscited by 12
- 2025SpeakStream: Streaming Text-to-Speech with Interleaved Datacited by 8
- 2025LLMVoX: Autoregressive Streaming Text-to-Speech Model for Any LLMcited by 7
- 2024Interleaved Speech-Text Language Models for Simple Streaming Text-to-Speech Synthesiscited by 6
- 2025StreamMel: Real-Time Zero-shot Text-to-Speech via Interleaved Continuous Autoregressive Modelingcited by 3
- 2025Pronunciation-Lexicon Free Training for Phoneme-based Crosslingual ASR via Joint Stochastic Approximationcited by 3
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026TRACE: Training-Free Partial Audio Deepfake Detection via Embedding Trajectory Analysis of Speech Foundation Modelscited by 5
- 2026X2-NativeCursor: Native-Token Text Progress Tracking for Incremental-Text Streaming Codec TTScited by 0
- 2026Evaluating Real-Time Voice Agents: From Component Quality to Grounded Outcomescited by 0
- 2026Evaluating Real-Time Voice Agents: From Component Quality to Grounded Outcomes25 Sep 2026
- 2026X2-NativeCursor: Native-Token Text Progress Tracking for Incremental-Text Streaming Codec TTS9 Sep 2026
- 2026TRACE: Training-Free Partial Audio Deepfake Detection via Embedding Trajectory Analysis of Speech Foundation Models1 Apr 2026
Similar, not cited
Close in wording, no citation either way
- 2026TADA: A Generative Framework for Speech Modeling via Text-Acoustic Dual Alignment
- 2026StreamAlign: Streaming Text-Aligned Speech Tokenization
- 2024Improving Robustness of LLM-based Speech Synthesis by Learning Monotonic Alignment
- 2026ARCHI-TTS: A flow-matching-based Text-to-Speech Model with Self-supervised Semantic Aligner and Accelerated Inference
- 2026TASU2: Controllable CTC Simulation for Alignment and Low-Resource Adaptation of Speech LLMs
- 2026Gradient-Based Speech-to-Text Alignment for Any ASR Model: From CTC to Speech LLMs
- 2025Data-efficient Targeted Token-level Preference Optimization for LLM-based Text-to-Speech
- 2026End-to-End Training for Discrete Token LLM based TTS System