# MoonCast: High-Quality Zero-Shot Podcast Generation

arXiv: https://arxiv.org/abs/2503.14345 · HTML: https://mlsanity.com/paper/2503.14345/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 42 references that are in the map.
- First posted on arXiv 18 Mar 2025.

## Builds on (most cited first, 30 of 37)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation](https://mlsanity.com/paper/1911.00536/) (2019)
- [Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers](https://mlsanity.com/paper/2301.02111/) (2023)
- [Vector-quantized Image Modeling with Improved VQGAN](https://mlsanity.com/paper/2110.04627/) (2021)
- [AudioLM: a Language Modeling Approach to Audio Generation](https://mlsanity.com/paper/2209.03143/) (2022)
- [Towards a Human-like Open-Domain Chatbot](https://mlsanity.com/paper/2001.09977/) (2020)
- [High-Fidelity Audio Compression with Improved RVQGAN](https://mlsanity.com/paper/2306.06546/) (2023)
- [Voicebox: Text-Guided Multilingual Universal Speech Generation at Scale](https://mlsanity.com/paper/2306.15687/) (2023)
- [W2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Training](https://mlsanity.com/paper/2108.06209/) (2021)
- [CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokens](https://mlsanity.com/paper/2407.05407/) (2024)
- [Seed-TTS: A Family of High-Quality Versatile Speech Generation Models](https://mlsanity.com/paper/2406.02430/) (2024)
- [CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Models](https://mlsanity.com/paper/2412.10117/) (2024)
- [Beyond Goldfish Memory: Long-Term Open-Domain Conversation](https://mlsanity.com/paper/2107.07567/) (2021)
- [BigVGAN: A Universal Neural Vocoder with Large-Scale Training](https://mlsanity.com/paper/2206.04658/) (2022)
- [NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizers](https://mlsanity.com/paper/2304.09116/) (2023)
- [NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Models](https://mlsanity.com/paper/2403.03100/) (2024)
- [Emilia: An Extensive, Multilingual, and Diverse Speech Dataset for Large-Scale Speech Generation](https://mlsanity.com/paper/2407.05361/) (2024)
- [Seamless: Multilingual Expressive and Streaming Speech Translation](https://mlsanity.com/paper/2312.05187/) (2023)
- [MaskGCT: Zero-Shot Text-to-Speech with Masked Generative Codec Transformer](https://mlsanity.com/paper/2409.00750/) (2024)
- [Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognition](https://mlsanity.com/paper/2206.08317/) (2022)
- [Generative Spoken Dialogue Language Modeling](https://mlsanity.com/paper/2203.16502/) (2022)
- [VALL-E 2: Neural Codec Language Models are Human Parity Zero-Shot Text to Speech Synthesizers](https://mlsanity.com/paper/2406.05370/) (2024)
- [SoundStorm: Efficient Parallel Audio Generation](https://mlsanity.com/paper/2305.09636/) (2023)
- [BASE TTS: Lessons from building a billion-parameter Text-to-Speech model on 100K hours of data](https://mlsanity.com/paper/2402.08093/) (2024)
- [Autoregressive Diffusion Transformer for Text-to-Speech Synthesis](https://mlsanity.com/paper/2406.05551/) (2024)
- [Towards human-like spoken dialogue generation between AI agents from written dialogue](https://mlsanity.com/paper/2310.01088/) (2023)

## Built on by (most cited first, 18 of 18)

- [Kimi-Audio Technical Report](https://mlsanity.com/paper/2504.18425/) (2025)
- [VibeVoice Technical Report](https://mlsanity.com/paper/2508.19205/) (2025)
- [Ming-Flash-Omni: A Sparse, Unified Architecture for Multimodal Perception and Generation](https://mlsanity.com/paper/2510.24821/) (2025)
- [CoVoMix2: Advancing Zero-Shot Dialogue Generation with Fully Non-Autoregressive Flow Matching](https://mlsanity.com/paper/2506.00885/) (2025)
- [ZipVoice-Dialog: Non-Autoregressive Spoken Dialogue Generation with Flow Matching](https://mlsanity.com/paper/2507.09318/) (2025)
- [VoiceBridge: General Speech Restoration with One-step Latent Bridge Models](https://mlsanity.com/paper/2509.25275/) (2025)
- [VoxServe: Streaming-Centric Serving System for Speech Language Models](https://mlsanity.com/paper/2602.00269/) (2026)
- [TELEVAL: A Benchmark Designed for Spoken Language Models in Chinese Interactive Scenarios](https://mlsanity.com/paper/2507.18061/) (2025)
- [Scaling Spoken Language Models with Syllabic Speech Tokenization](https://mlsanity.com/paper/2509.26634/) (2025)
- [Vox-Evaluator: Enhancing Stability and Fidelity for Zero-shot TTS with A Multi-Level Evaluator](https://mlsanity.com/paper/2510.20210/) (2025)
- [TAVID: Text-Driven Audio-Visual Interactive Dialogue Generation](https://mlsanity.com/paper/2512.20296/) (2025)
- [PodBench: A Comprehensive Benchmark for Instruction-Aware Audio-Oriented Podcast Script Generation](https://mlsanity.com/paper/2601.14903/) (2026)
- [Context-Aware Multimodal Claim Verification in Spoken Dialogues](https://mlsanity.com/paper/2606.11420/) (2026)
- [Bagpiper-TTS: Natural Language Guided Universal Speech Synthesis](https://mlsanity.com/paper/2606.22811/) (2026)
- [SpeakerSleuth: Can Large Audio-Language Models Judge Speaker Consistency across Multi-turn Dialogues?](https://mlsanity.com/paper/2601.04029/) (2026)
- [Reference-Driven Multi-Speaker Audio Scene Generation from In-the-Wild Priors](https://mlsanity.com/paper/2606.19325/) (2026)
- [On Improving Faithfulness of Podcasts from Documents](https://mlsanity.com/paper/2607.21961/) (2026)
- [Alignment-Free Text-Audiobox for Voice Dubbing and Full-Duplex Dialogue Synthesis](https://mlsanity.com/paper/2609.03992/) (2026)

## Abstract (arXiv)

Recent advances in text-to-speech synthesis have achieved notable success in generating high-quality short utterances for individual speakers. However, these systems still face challenges when extending their capabilities to long, multi-speaker, and spontaneous dialogues, typical of real-world scenarios such as podcasts. These limitations arise from two primary challenges: 1) long speech: podcasts typically span several minutes, exceeding the upper limit of most existing work; 2) spontaneity: podcasts are marked by their spontaneous, oral nature, which sharply contrasts with formal, written contexts; existing works often fall short in capturing this spontaneity. In this paper, we propose MoonCast, a solution for high-quality zero-shot podcast generation, aiming to synthesize natural podcast-style speech from text-only sources (e.g., stories, technical reports, news in TXT, PDF, or Web URL formats) using the voices of unseen speakers. To generate long audio, we adopt a long-context language model-based audio modeling approach utilizing large-scale long-context speech data. To enhance spontaneity, we utilize a podcast generation module to generate scripts with spontaneous details, which have been empirically shown to be as crucial as the text-to-speech modeling itself. Experiments demonstrate that MoonCast outperforms baselines, with particularly notable improvements in spontaneity and coherence.