PilotTTS: A Disciplined Modular Recipe for Competitive Speech Synthesis
Not on the map. Why?
Abstract (arXiv)
Building state-of-the-art text-to-speech (TTS) systems typically demands millions of hours of proprietary data and complex multi-stage architectures, creating substantial barriers for resource-constrained research teams. In this report, we present PilotTTS, a lightweight autoregressive TTS system that achieves competitive performance through minimalist architecture and rigorous data engineering. PilotTTS is trained on only 200K hours of data processed entirely with open-source tools. Specifically, our contributions are: (1) a reproducible multi-stage data processing pipeline covering quality assessment, label annotation, and filtering, and (2) a compact model architecture that employs Q-Former-based conditioning to decouple speaker identity from speaking style via cross-sample paired training. Within a unified framework, PilotTTS supports zero-shot voice cloning, emotion synthesis (11 categories), paralinguistic synthesis (4 categories), and Chinese dialect synthesis (14 dialects). On the Seed-TTS Eval benchmark, PilotTTS achieves the lowest WER of 1.50% on test-en, a CER of 0.87% on test-zh, and the highest speaker similarity on both test sets (0.862 and 0.815), outperforming systems trained on significantly larger datasets. We release the complete data pipeline recipe, pretrained weights, and code at https://github.com/AMAPVOICE/PilotTTS.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 31 of its 32 references that are in the map. First posted on arXiv 26 May 2026.
Builds on
31 of 32 references.
References · 31
- 2025Qwen3 Technical Reportcited by 8,820
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2022Flow Matching for Generative Modelingcited by 5,702
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
- 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,154
- 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 752
- 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matchingcited by 361
- 2021W2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Trainingcited by 347
- 2024CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokenscited by 334
- 2024Seed-TTS: A Family of High-Quality Versatile Speech Generation Modelscited by 300
- 2024CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Modelscited by 288
19 more
- 2020DNSMOS: A Non-Intrusive Perceptual Objective Speech Quality metric to evaluate Noise Suppressorscited by 241
- 2023NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizerscited by 235
- 2024NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Modelscited by 226
- 2025CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-trainingcited by 150
- 2022Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognitioncited by 144
- 2026Qwen3-TTS Technical Reportcited by 118
- 2026Qwen3-ASR Technical Reportcited by 118
- 2023Powerset multi-class cross entropy loss for neural speaker diarizationcited by 111
- 2023CAM++: A Fast and Efficient Network for Speaker Verification Using Context-Aware Maskingcited by 102
- 2025DiTAR: Diffusion Transformer Autoregressive Modeling for Speech Generationcited by 57
- 2025MiniMax-Speech: Intrinsic Zero-Shot Text-to-Speech with a Learnable Speaker Encodercited by 48
- 2025FireRedASR: Open-Source Industrial-Grade Mandarin Speech Recognition Models from Encoder-Decoder to LLM Integrationcited by 46
- 2025VoxCPM: Tokenizer-Free TTS for Context-Aware Speech Generation and True-to-Life Voice Cloningcited by 42
- 2025FireRedTTS-2: Towards Long Conversational Speech Generation for Podcast and Chatbotcited by 35
- 2025VibeVoice Technical Reportcited by 29
- 2025IndexTTS: An Industrial-Level Controllable and Efficient Zero-Shot Text-To-Speech Systemcited by 28
- 20243D-Speaker-Toolkit: An Open-Source Toolkit for Multimodal Speaker Verification and Diarizationcited by 16
- 2024Towards Efficient Visual-Language Alignment of the Q-Former for Visual Reasoning Taskscited by 9
- 2026Fish Audio S2 Technical Reportcited by 9
Built on by
Not cited yet.