# CTC-TTS: LLM-based dual-streaming text-to-speech with CTC alignment

arXiv: https://arxiv.org/abs/2602.19574 · HTML: https://mlsanity.com/paper/2602.19574/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 28 references that are in the map.
- First posted on arXiv 23 Feb 2026.

## Builds on (most cited first, 22 of 22)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [SoundStream: An End-to-End Neural Audio Codec](https://mlsanity.com/paper/2107.03312/) (2021)
- [Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers](https://mlsanity.com/paper/2301.02111/) (2023)
- [High Fidelity Neural Audio Compression](https://mlsanity.com/paper/2210.13438/) (2022)
- [UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022](https://mlsanity.com/paper/2204.02152/) (2022)
- [CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokens](https://mlsanity.com/paper/2407.05407/) (2024)
- [Seed-TTS: A Family of High-Quality Versatile Speech Generation Models](https://mlsanity.com/paper/2406.02430/) (2024)
- [CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Models](https://mlsanity.com/paper/2412.10117/) (2024)
- [Mini-Omni: Language Models Can Hear, Talk While Thinking in Streaming](https://mlsanity.com/paper/2408.16725/) (2024)
- [WavTokenizer: an Efficient Acoustic Discrete Codec Tokenizer for Audio Language Modeling](https://mlsanity.com/paper/2408.16532/) (2024)
- [ELLA-V: Stable Neural Codec Language Modeling with Alignment-guided Sequence Reordering](https://mlsanity.com/paper/2401.07333/) (2024)
- [Whistle: Data-Efficient Multilingual and Crosslingual Speech Recognition via Weakly Phonetic Supervision](https://mlsanity.com/paper/2406.02166/) (2024)
- [Less Peaky and More Accurate CTC Forced Alignment by Label Priors](https://mlsanity.com/paper/2406.02560/) (2024)
- [SpeakStream: Streaming Text-to-Speech with Interleaved Data](https://mlsanity.com/paper/2505.19206/) (2025)
- [LLMVoX: Autoregressive Streaming Text-to-Speech Model for Any LLM](https://mlsanity.com/paper/2503.04724/) (2025)
- [Interleaved Speech-Text Language Models for Simple Streaming Text-to-Speech Synthesis](https://mlsanity.com/paper/2412.16102/) (2024)
- [StreamMel: Real-Time Zero-shot Text-to-Speech via Interleaved Continuous Autoregressive Modeling](https://mlsanity.com/paper/2506.12570/) (2025)
- [Pronunciation-Lexicon Free Training for Phoneme-based Crosslingual ASR via Joint Stochastic Approximation](https://mlsanity.com/paper/2507.06249/) (2025)

## Built on by (most cited first, 3 of 3)

- [TRACE: Training-Free Partial Audio Deepfake Detection via Embedding Trajectory Analysis of Speech Foundation Models](https://mlsanity.com/paper/2604.01083/) (2026)
- [X2-NativeCursor: Native-Token Text Progress Tracking for Incremental-Text Streaming Codec TTS](https://mlsanity.com/paper/2609.09677/) (2026)
- [Evaluating Real-Time Voice Agents: From Component Quality to Grounded Outcomes](https://mlsanity.com/paper/2609.30798/) (2026)

## Abstract (arXiv)

Large-language-model (LLM)-based text-to-speech (TTS) systems can generate natural speech, but most are not designed for low-latency dual-streaming synthesis. High-quality dual-streaming TTS depends on accurate text--speech alignment and well-designed training sequences that balance synthesis quality and latency. Prior work often relies on GMM-HMM based forced-alignment toolkits (e.g., MFA), which are pipeline-heavy and less flexible than neural aligners; fixed-ratio interleaving of text and speech tokens struggles to capture text--speech alignment regularities. We propose CTC-TTS, which replaces MFA with a CTC based aligner and introduces a bi-word based interleaving strategy. Two variants are designed: CTC-TTS-L (token concatenation along the sequence length) for higher quality and CTC-TTS-F (embedding stacking along the feature dimension) for lower latency. Experiments show that CTC-TTS outperforms fixed-ratio interleaving and MFA-based baselines on streaming synthesis and zero-shot tasks.