# VoxCPM: Tokenizer-Free TTS for Context-Aware Speech Generation and True-to-Life Voice Cloning

arXiv: https://arxiv.org/abs/2509.24650 · HTML: https://mlsanity.com/paper/2509.24650/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Sep 2025.

## Built on by (most cited first, 30 of 42)

- [Qwen3-TTS Technical Report](https://mlsanity.com/paper/2601.15621/) (2026)
- [OmniVoice: Towards Omnilingual Zero-Shot Text-to-Speech with Diffusion Language Models](https://mlsanity.com/paper/2604.00688/) (2026)
- [MOSS-TTS Technical Report](https://mlsanity.com/paper/2603.18090/) (2026)
- [IndexTTS 2.5 Technical Report](https://mlsanity.com/paper/2601.03888/) (2026)
- [dots.tts Technical Report](https://mlsanity.com/paper/2606.07080/) (2026)
- [SoulX-Singer: Towards High-Quality Zero-Shot Singing Voice Synthesis](https://mlsanity.com/paper/2602.07803/) (2026)
- [LongCat-AudioDiT: High-Fidelity Diffusion Text-to-Speech in the Waveform Latent Space](https://mlsanity.com/paper/2603.29339/) (2026)
- [Chatterbox-Flash: Prior-Calibrated Block Diffusion for Streaming Zero-Shot TTS](https://mlsanity.com/paper/2605.30748/) (2026)
- [Unifying Speech Recognition, Synthesis and Conversion with Autoregressive Transformers](https://mlsanity.com/paper/2601.10770/) (2026)
- [Video-Robin: Autoregressive Diffusion Planning for Intent-Grounded Video-to-Music Generation](https://mlsanity.com/paper/2604.17656/) (2026)
- [TTS-PRISM: A Perceptual Reasoning and Interpretable Speech Model for Fine-Grained Diagnosis](https://mlsanity.com/paper/2604.22225/) (2026)
- [JaiTTS: A Thai Voice Cloning Model](https://mlsanity.com/paper/2604.27607/) (2026)
- [WavTTS: Towards High-Quality Zero-Shot TTS via Direct Raw Waveform Modeling](https://mlsanity.com/paper/2606.03455/) (2026)
- [AMR: Adaptive Modality Routing for Multimodal Polyglot Speaker Identification](https://mlsanity.com/paper/2606.29335/) (2026)
- [DELTA-TTS: Adapting Autoregressive Model into Diffusion Language Model for Text-to-Speech](https://mlsanity.com/paper/2607.04140/) (2026)
- [What Counts as Real? Speech Restoration and Voice Quality Conversion Pose New Challenges to Deepfake Detection](https://mlsanity.com/paper/2603.14033/) (2026)
- [Conditioned in Training, Dispensable at Inference: Typological Conditioning for Multilingual Speech Translation](https://mlsanity.com/paper/2605.16026/) (2026)
- [RobustSpeechFlow: Learning Robust Text-to-Speech Trajectories via Augmentation-based Contrastive Flow Matching](https://mlsanity.com/paper/2605.22083/) (2026)
- [PilotTTS: A Disciplined Modular Recipe for Competitive Speech Synthesis](https://mlsanity.com/paper/2605.27258/) (2026)
- [MELD: Mel-Spectrogram-Based Speech Language Modeling with Discrete Latent Variables](https://mlsanity.com/paper/2605.29859/) (2026)
- [Closing the Quality Gap in Low-Resource Text-to-Speech: LoRA Fine-Tuning of VoxCPM2 for Khmer and Korean](https://mlsanity.com/paper/2606.26618/) (2026)
- [BlueMagpie-TTS: A Token-Efficient Tokenizer, Language Model, and TTS for Taiwanese-Accent Code-Switching Speech](https://mlsanity.com/paper/2607.06054/) (2026)
- [ReGen: Hierarchical Multi-Prompt Representation Generation for Efficient Waveform Diffusion Models](https://mlsanity.com/paper/2607.09134/) (2026)
- [FreyaTTS: A Compact Tokenizer-Free Flow-Matching Transformer for Turkish-First Speech Synthesis](https://mlsanity.com/paper/2607.09530/) (2026)
- [VoxENES 2026: Benchmarking Generalization of Speech Spoofing Detectors Against LLM-Era TTS and Voice Conversion](https://mlsanity.com/paper/2607.11706/) (2026)
- [Time-Frequency Consistency Learning for Robust Speech Deepfake Detection](https://mlsanity.com/paper/2607.17761/) (2026)
- [Let Me Look at You: Advanced Facial Expression Modeling for Conversational Speech Synthesis](https://mlsanity.com/paper/2607.24430/) (2026)
- [Zero-Shot Face-to-Speech Synthesis via Latent Space Adaptation of a Style-Diffusion TTS Model](https://mlsanity.com/paper/2607.26742/) (2026)
- [Stable Autoregressive Speech Generation with Low-Frame-Rate High-Dimensional Continuous Tokens](https://mlsanity.com/paper/2607.29363/) (2026)
- [CuteTTS: Efficient and High-Quality Speech Synthesis via Autoregressive Modeling of Continuous Latents](https://mlsanity.com/paper/2608.08638/) (2026)

## Abstract (arXiv)

Generative models for speech synthesis face a fundamental trade-off: discrete tokens ensure stability but sacrifice expressivity, while continuous signals retain acoustic richness but suffer from error accumulation due to task entanglement. This challenge has driven the field towards multi-stage pipelines that rely on pre-trained speech tokenizers, but these create a semantic-acoustic divide, limiting holistic and expressive speech generation. We resolve these dilemma through hierarchical semantic-acoustic modeling with semi-discrete residual representations and present a novel tokenizer-free TTS model VoxCPM. Our framework introduces a differentiable quantization bottleneck that induces natural specialization: a Text-Semantic Language Model (TSLM) generates semantic-prosodic plans, while a Residual Acoustic Model (RALM) recovers fine-grained acoustic details. This hierarchical semantic-acoustic representation guides a local diffusion-based decoder to generate high-fidelity speech latents. Critically, the entire architecture is trained end-to-end under a simple diffusion objective, eliminating dependency on external speech tokenizers. Trained on a massive 1.8 million hours of bilingual corpus, our VoxCPM-0.5B model achieves state-of-the-art zero-shot TTS performance among open-source systems, demonstrating that our approach delivers expressive and stable synthesis. Besides, VoxCPM shows the capability to comprehend text to infer and generate appropriate prosody and style, delivering speech with context-aware expressiveness and natural flow. To facilitate community-driven research and development, VoxCPM is publicly accessible under Apache 2.0.