PodAgent: A Comprehensive Framework for Podcast Generation
Not on the map. Why?
Abstract (arXiv)
Existing Existing automatic audio generation methods struggle to generate podcast-like audio programs effectively. The key challenges lie in in-depth content generation, appropriate and expressive voice production. This paper proposed PodAgent, a comprehensive framework for creating audio programs. PodAgent 1) generates informative topic-discussion content by designing a Host-Guest-Writer multi-agent collaboration system, 2) builds a voice pool for suitable voice-role matching and 3) utilizes LLM-enhanced speech synthesis method to generate expressive conversational speech. Given the absence of standardized evaluation criteria for podcast-like audio generation, we developed comprehensive assessment guidelines to effectively evaluate the model's performance. Experimental results demonstrate PodAgent's effectiveness, significantly surpassing direct GPT-4 generation in topic-discussion dialogue content, achieving an 87.4% voice-matching accuracy, and producing more expressive speech through LLM-guided synthesis. Demo page: https://podcast-agent.github.io/demo/. Source code: https://github.com/yujxx/PodAgent.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 32 of its 52 references that are in the map. First posted on arXiv 1 Mar 2025.
Builds on
32 of 52 references.
References · 32
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2023GPT-4 Technical Reportcited by 18,132
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
- 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
- 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,134
- 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,068
- 2023Large Language Models are not Fair Evaluatorscited by 1,050
- 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,002
- 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 995
20 more
- 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 867
- 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debatecited by 724
- 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 639
- 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 611
- 2023SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilitiescited by 518
- 2023AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretrainingcited by 343
- 2023Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agentscited by 329
- 2023Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Modelscited by 323
- 2024CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokenscited by 323
- 2021YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyonecited by 277
- 2023AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Headcited by 264
- 2024LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individualscited by 235
- 2024AnyGPT: Unified Multimodal LLM with Discrete Sequence Modelingcited by 213
- 2022NaturalSpeech: End-to-End Text to Speech Synthesis with Human-Level Qualitycited by 151
- 2022PromptTTS: Controllable Text-to-Speech with Text Descriptionscited by 106
- 2023LibriTTS-R: A Restored Multi-Speaker Text-to-Speech Corpuscited by 100
- 2023InstructTTS: Modelling Expressive TTS in Discrete Latent Space with Natural Language Style Promptcited by 90
- 2023A Comprehensive Analysis of the Effectiveness of Large Language Models as Automatic Dialogue Evaluatorscited by 45
- 2024SpeechCraft: A Fine-grained Expressive Speech Dataset with Natural Language Descriptioncited by 20
- 2023StyleCap: Automatic Speaking-Style Captioning from Speech Based on Speech and Language Self-supervised Learning Modelscited by 7
Built on by
Citing papers per year: 2025: 1; 2026: 2.
3 papers.