# Better speech synthesis through scaling

arXiv: https://arxiv.org/abs/2305.07243 · HTML: https://mlsanity.com/paper/2305.07243/

## Key facts

- Cited by 63 papers in the map (data through 14 Oct 2026).
- Placed in Audio, speech and music, one of 14 directions in Diffusion models.
- Builds on 7 of its 20 references that are in the map.
- First posted on arXiv 12 May 2023.

## Placed in

- Diffusion models / [Audio, speech and music](https://mlsanity.com/diffusion/audio-speech-and-music/)

## Builds on (most cited first, 7 of 7)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speech](https://mlsanity.com/paper/1904.02882/) (2019)
- [Hi-Fi Multi-Speaker English TTS Dataset](https://mlsanity.com/paper/2104.01497/) (2021)

## Built on by (most cited first, 30 of 63)

- [CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokens](https://mlsanity.com/paper/2407.05407/) (2024)
- [XTTS: a Massively Multilingual Zero-Shot Text-to-Speech Model](https://mlsanity.com/paper/2406.04904/) (2024)
- [MaskGCT: Zero-Shot Text-to-Speech with Masked Generative Codec Transformer](https://mlsanity.com/paper/2409.00750/) (2024)
- [Matcha-TTS: A fast TTS architecture with conditional flow matching](https://mlsanity.com/paper/2309.03199/) (2023)
- [Step-Audio 2 Technical Report](https://mlsanity.com/paper/2507.16632/) (2025)
- [Recent Advances in Speech Language Models: A Survey](https://mlsanity.com/paper/2410.03751/) (2024)
- [BASE TTS: Lessons from building a billion-parameter Text-to-Speech model on 100K hours of data](https://mlsanity.com/paper/2402.08093/) (2024)
- [Recent Advances in Discrete Speech Tokens: A Review](https://mlsanity.com/paper/2502.06490/) (2025)
- [HierSpeech++: Bridging the Gap between Semantic and Acoustic Representation of Speech by Hierarchical Variational Inference for Zero-shot Speech Synthesis](https://mlsanity.com/paper/2311.12454/) (2023)
- [InteractiveOmni: A Unified Omni-modal Model for Audio-Visual Multi-turn Dialogue](https://mlsanity.com/paper/2510.13747/) (2025)
- [IndexTTS: An Industrial-Level Controllable and Efficient Zero-Shot Text-To-Speech System](https://mlsanity.com/paper/2502.05512/) (2025)
- [Enhancing Zero-shot Text-to-Speech Synthesis with Human Feedback](https://mlsanity.com/paper/2406.00654/) (2024)
- [MOSS-TTS Technical Report](https://mlsanity.com/paper/2603.18090/) (2026)
- [Tree Attention: Topology-aware Decoding for Long-Context Attention on GPU clusters](https://mlsanity.com/paper/2408.04093/) (2024)
- [I Can Hear You: Selective Robust Training for Deepfake Audio Detection](https://mlsanity.com/paper/2411.00121/) (2024)
- [Robust Zero-Shot Text-to-Speech Synthesis with Reverse Inference Optimization](https://mlsanity.com/paper/2407.02243/) (2024)
- [Harder or Different? Understanding Generalization of Audio Deepfake Detection](https://mlsanity.com/paper/2406.03512/) (2024)
- [GenVC: Self-Supervised Zero-Shot Voice Conversion](https://mlsanity.com/paper/2502.04519/) (2025)
- [TTSDS2: Resources and Benchmark for Evaluating Human-Quality Text to Speech Systems](https://mlsanity.com/paper/2506.19441/) (2025)
- [Enhancing the Stability of LLM-based Speech Generation Systems through Self-Supervised Representations](https://mlsanity.com/paper/2402.03407/) (2024)
- [Takin: A Cohort of Superior Quality Zero-shot Speech Generation Models](https://mlsanity.com/paper/2409.12139/) (2024)
- [Asynchronous Reasoning: Training-Free Interactive Thinking LLMs](https://mlsanity.com/paper/2512.10931/) (2025)
- [Small-E: Small Language Model with Linear Attention for Efficient Speech Synthesis](https://mlsanity.com/paper/2406.04467/) (2024)
- [SafeSpeech: Robust and Universal Voice Protection Against Malicious Speech Synthesis](https://mlsanity.com/paper/2504.09839/) (2025)
- [Fake it to make it: Using synthetic data to remedy the data shortage in joint multimodal speech-and-gesture synthesis](https://mlsanity.com/paper/2404.19622/) (2024)
- [VoiceWukong: Benchmarking Deepfake Voice Detection](https://mlsanity.com/paper/2409.06348/) (2024)
- [Voice Cloning: Comprehensive Survey](https://mlsanity.com/paper/2505.00579/) (2025)
- [Custom Data Augmentation for low resource ASR using Bark and Retrieval-Based Voice Conversion](https://mlsanity.com/paper/2311.14836/) (2023)
- [PSCodec: A Series of High-Fidelity Low-bitrate Neural Speech Codecs Leveraging Prompt Encoders](https://mlsanity.com/paper/2404.02702/) (2024)
- [Improving Audio Codec-based Zero-Shot Text-to-Speech Synthesis with Multi-Modal Context and Large Language Model](https://mlsanity.com/paper/2406.03706/) (2024)

## Abstract (arXiv)

In recent years, the field of image generation has been revolutionized by the application of autoregressive transformers and DDPMs. These approaches model the process of image generation as a step-wise probabilistic processes and leverage large amounts of compute and data to learn the image distribution. This methodology of improving performance need not be confined to images. This paper describes a way to apply advances in the image generative domain to speech synthesis. The result is TorToise -- an expressive, multi-voice text-to-speech system. All model code and trained weights have been open-sourced at https://github.com/neonbjb/tortoise-tts.