MUSE: A Multi-agent Framework for Unconstrained Story Envisioning via Closed-Loop Cognitive Orchestration
Not on the map. Why?
Abstract (arXiv)
Generating long-form audio-visual stories from a short user prompt remains challenging due to an intent-execution gap, where high-level narrative intent must be preserved across coherent, shot-level multimodal generation over long horizons. Existing approaches typically rely on feed-forward pipelines or prompt-only refinement, which often leads to semantic drift and identity inconsistency as sequences grow longer. We address this challenge by formulating storytelling as a closed-loop constraint enforcement problem and propose MUSE, a multi-agent framework that coordinates generation through an iterative plan-execute-verify-revise loop. MUSE translates narrative intent into explicit, machine-executable controls over identity, spatial composition, and temporal continuity, and applies targeted multimodal feedback to correct violations during generation. To evaluate open-ended storytelling without ground-truth references, we introduce MUSEBench, a reference-free evaluation protocol validated by human judgments. Experiments demonstrate that MUSE substantially improves long-horizon narrative coherence, cross-modal identity consistency, and cinematic quality compared with representative baselines.
Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 36 of its 43 references that are in the map. First posted on arXiv 3 Feb 2026.
Builds on
36 of 43 references.
References · 36
- 2023GPT-4 Technical Reportcited by 18,132
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
- 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
- 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
- 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
- 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
- 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
- 2023ChatDev: Communicative Agents for Software Developmentcited by 893
- 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 737
- 2023AudioLDM: Text-to-Audio Generation with Latent Diffusion Modelscited by 592
- 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 467
24 more
- 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matchingcited by 353
- 2024CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Modelscited by 282
- 2021YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyonecited by 277
- 2024StoryDiffusion: Consistent Self-Attention for Long-Range Image and Video Generationcited by 240
- 2024ConsistI2V: Enhancing Visual Consistency for Image-to-Video Generationcited by 130
- 2022PromptTTS: Controllable Text-to-Speech with Text Descriptionscited by 106
- 2024Large Multimodal Agents: A Surveycited by 99
- 2025Automated Movie Generation via Multi-Agent CoT Planningcited by 82
- 2022NUWA-Infinity: Autoregressive over Autoregressive Generation for Infinite Visual Synthesiscited by 71
- 2024Vlogger: Make Your Dream A Vlogcited by 70
- 2025VideoWorld: Exploring Knowledge Learning from Unlabeled Videoscited by 42
- 2024Anim-Director: A Large Multimodal Model Powered Agent for Controllable Animation Video Generationcited by 35
- 2025VideoGen-of-Thought: Step-by-step generating multi-shot video with minimal manual interventioncited by 33
- 2024StoryAgent: Customized Storytelling Video Generation via Multi-Agent Collaborationcited by 28
- 2025MAViS: A Multi-Agent Framework for Long-Sequence Video Storytellingcited by 19
- 2024GRPose: Learning Graph Relations for Human Image Generation with Pose Priorscited by 14
- 2025MoEE: Mixture of Emotion Experts for Audio-Driven Portrait Animationcited by 12
- 2025MotionStrata: Hierarchical Motion Latents for Compact Video Autoencodingcited by 9
- 2024Video Is Worth a Thousand Images: Exploring the Latest Trends in Long Video Generationcited by 7
- 2025A Survey on Long-Video Storytelling Generation: Architectures, Consistency, and Cinematic Qualitycited by 6
- 2025VinaBench: Benchmark for Faithful and Consistent Visual Narrativescited by 4
- 2025Structure-guided Diffusion Transformer for Low-Light Image Enhancementcited by 4
- 2025Character-Centered Dialogue Generation from Scene-Level Promptscited by 2
- 2025Audiobook-CC: Controllable Long-context Speech Generation for Multicast Audiobookcited by 1
Built on by
Citing papers per year: 2026: 5.
5 papers.
- 2026GroundShot: Visually Consistent Multi-Shot Long Video Generation via Entity-Grounded Shot Schedulingcited by 4
- 2026EvoState: Closed-Loop Visual State Management for Long-Form Video Generationcited by 2
- 2026FilmWorld: Agentic Novel-to-Film Generation through Dynamic Cinematic World Modelingcited by 2
- 2026RiO-DETR: DETR for Real-time Oriented Object Detectioncited by 1
- 2026Stable Curves, Unstable Items: Item-Level Scaling Heterogeneity in Video LLMscited by 0
- 2026Stable Curves, Unstable Items: Item-Level Scaling Heterogeneity in Video LLMs7 Aug 2026
- 2026FilmWorld: Agentic Novel-to-Film Generation through Dynamic Cinematic World Modeling21 Jul 2026
- 2026GroundShot: Visually Consistent Multi-Shot Long Video Generation via Entity-Grounded Shot Scheduling18 Jun 2026
- 2026EvoState: Closed-Loop Visual State Management for Long-Form Video Generation15 Jun 2026
- 2026RiO-DETR: DETR for Real-time Oriented Object Detection10 Mar 2026