Bridging Your Imagination with Audio-Video Generation via a Unified Director
Not on the map. Why?
Abstract (arXiv)
Existing AI-driven video creation systems typically treat script drafting and key-shot design as two disjoint tasks: the former relies on large language models, while the latter depends on image generation models. We argue that these two tasks should be unified within a single framework, as logical reasoning and imaginative thinking are both fundamental qualities of a film director. In this work, we propose UniMAGE, a unified director model that bridges user prompts with well-structured scripts, thereby empowering non-experts to produce long-context, multi-shot films by leveraging existing audio-video generation models. To achieve this, we employ the Mixture-of-Transformers architecture that unifies text and image generation. To further enhance narrative logic and keyframe consistency, we introduce a ``first interleaving, then disentangling'' training paradigm. Specifically, we first perform Interleaved Concept Learning, which utilizes interleaved text-image data to foster the model's deeper understanding and imaginative interpretation of scripts. We then conduct Disentangled Expert Learning, which decouples script writing from keyframe generation, enabling greater flexibility and creativity in storytelling. Extensive experiments demonstrate that UniMAGE achieves state-of-the-art performance among open-source models, generating logically coherent video scripts and visually consistent keyframe images.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 31 of its 35 references that are in the map. First posted on arXiv 29 Dec 2025.
Builds on
31 of 35 references.
References · 31
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
- 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
- 2024Qwen2.5-Coder Technical Reportcited by 1,203
- 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,188
- 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
- 2025Emerging Properties in Unified Multimodal Pretrainingcited by 897
- 2024Emu3: Next-Token Prediction is All You Needcited by 781
- 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 752
- 2023Scaling Vision Transformers to 22 Billion Parameterscited by 694
- 2024Large Language Models: A Surveycited by 525
- 2023Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levelscited by 503
19 more
- 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 483
- 2025Seedance 1.0: Exploring the Boundaries of Video Generation Modelscited by 259
- 2024StoryDiffusion: Consistent Self-Attention for Long-Range Image and Video Generationcited by 247
- 2023Patch n' Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolutioncited by 229
- 2022MM-Diffusion: Learning Multi-Modal Diffusion Models for Joint Audio and Video Generationcited by 227
- 2025GoT: Unleashing Reasoning Capability of Multimodal Large Language Model for Visual Generation and Editingcited by 104
- 2025Automated Movie Generation via Multi-Agent CoT Planningcited by 89
- 2024SEED-Story: Multimodal Long Story Generation with Large Language Modelcited by 73
- 2024VideoStudio: Generating Consistent-Content and Multi-Scene Videoscited by 64
- 2025UniVerse-1: Unified Audio-Video Generation via Stitching of Expertscited by 61
- 2025Captain Cinema: Towards Short Movie Generationcited by 48
- 2024Anim-Director: A Large Multimodal Model Powered Agent for Controllable Animation Video Generationcited by 37
- 2025FilmAgent: A Multi-Agent Framework for End-to-End Film Automation in Virtual 3D Spacescited by 28
- 2025FilMaster: Bridging Cinematic Principles and Generative AI for Automated Film Generationcited by 28
- 2024SyncFlow: Toward Temporally Aligned Joint Audio-Video Generation from Textcited by 26
- 2025UniForm: A Unified Multi-Task Diffusion Transformer for Audio-Video Generationcited by 22
- 2024TheaterGen: Character Management with LLM for Consistent Multi-turn Image Generationcited by 19
- 2025Story2Board: A Training-Free Approach for Expressive Storyboard Generationcited by 17
- 2024Story-Iter: A Training-free Iterative Paradigm for Long Story Visualizationcited by 13
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026The Script is All You Need: An Agentic Framework for Long-Horizon Dialogue-to-Cinematic Video Generation
- 2025Prompt-Driven Agentic Video Editing System: Autonomous Comprehension of Long-Form, Story-Driven Media
- 2026CamPilot: A Multi-Agent Cinematic Assistant for Camera-Controlled Movie Generation
- 2026CogOmniControl: Reasoning-Driven Controllable Video Generation via Creative Intent Cognition
- 2025Generative AI for Film Creation: A Survey of Recent Advances
- 2026Bridging Creative Intent and Visual Quality: Creator-Driven Recurrent Video Generation with Agentic Feedback Loops
- 2026TAKE 85: Testing Audiovisual filmmaKer's intEnt across 85 Hours of Film
- 2026Camera Artist: A Multi-Agent Framework for Cinematic Language Storytelling Video Generation