ShotVerse: Advancing Cinematic Camera Control for Text-Driven Multi-Shot Video Creation
Not on the map. Why?
Abstract (arXiv)
Text-driven video generation has democratized film creation, but camera control in cinematic multi-shot scenarios remains a significant block. Implicit textual prompts lack precision, while explicit trajectory conditioning imposes prohibitive manual overhead and often triggers execution failures in current models. To overcome this bottleneck, we propose a data-centric paradigm shift, positing that aligned (Caption, Trajectory, Video) triplets form an inherent joint distribution that can connect automated plotting and precise execution. Guided by this insight, we present ShotVerse, a ``Plan-then-Control'' framework that decouples generation into two collaborative agents: a VLM (Vision-Language Model)-based Planner that leverages spatial priors to obtain cinematic, globally aligned trajectories from text, and a Controller that renders these trajectories into multi-shot video content via a camera adapter. Central to our approach is the construction of a data foundation: we design an automated multi-shot camera calibration pipeline aligns disjoint single-shot trajectories into a unified global coordinate system. This facilitates the curation of ShotVerse-Bench, a high-fidelity cinematic dataset with a three-track evaluation protocol that serves as the bedrock for our framework. Extensive experiments demonstrate that ShotVerse effectively bridges the gap between unreliable textual control and labor-intensive manual plotting, achieving superior cinematic aesthetics and generating multi-shot videos that are both camera-accurate and cross-shot consistent.
Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 52 of its 70 references that are in the map. First posted on arXiv 12 Mar 2026.
Builds on
52 of 70 references.
References · 52
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
- 2022Scalable Diffusion Models with Transformerscited by 5,931
- 2022Flow Matching for Generative Modelingcited by 5,426
- 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
- 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
- 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
- 2023PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallelcited by 641
- 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 568
38 more
- 2023DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Visioncited by 514
- 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 488
- 2024CameraCtrl: Enabling Camera Control for Text-to-Video Generationcited by 448
- 2024SpatialRGPT: Grounded Spatial Reasoning in Vision Language Modelscited by 297
- 2025π3: Permutation-Equivariant Visual Geometry Learningcited by 292
- 2025ReCamMaster: Camera-Controlled Generative Rendering from A Single Videocited by 278
- 2025Seedance 1.0: Exploring the Boundaries of Video Generation Modelscited by 247
- 2024StoryDiffusion: Consistent Self-Attention for Long-Range Image and Video Generationcited by 240
- 2023MVImgNet: A Large-scale Dataset of Multi-view Imagescited by 219
- 2024VD3D: Taming Large Video Diffusion Transformers for 3D Camera Controlcited by 156
- 2024AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformerscited by 151
- 2024Collaborative Video Diffusion: Consistent Multi-video Generation with Camera Controlcited by 117
- 2024CamI2V: Camera-Controlled Image-to-Video Diffusion Modelcited by 115
- 2025Long Context Tuning for Video Generationcited by 110
- 2025CameraCtrl II: Dynamic Scene Exploration via Camera-controlled Video Diffusion Modelscited by 97
- 2024SynCamMaster: Synchronizing Multi-Camera Video Generation from Diverse Viewpointscited by 92
- 2025Mixture of Contexts for Long Video Generationcited by 90
- 2025Automated Movie Generation via Multi-Agent CoT Planningcited by 82
- 2025Scaling Spatial Intelligence with Multimodal Foundation Modelscited by 75
- 2025CineMaster: A 3D-Aware and Controllable Framework for Cinematic Text-to-Video Generationcited by 73
- 2025HoloCine: Holistic Generation of Cinematic Multi-Shot Long Video Narrativescited by 51
- 2024Director3D: Real-world Camera Trajectory and 3D Scene Generation from Textcited by 50
- 2025MotionCanvas: Cinematic Shot Design with Controllable Image-to-Video Generationcited by 50
- 2025Captain Cinema: Towards Short Movie Generationcited by 48
- 2025ShotAdapter: Text-to-Multi-Shot Video Generation with Diffusion Modelscited by 37
- 2025StoryMem: Multi-shot Long Video Storytelling with Memorycited by 35
- 2025VideoGen-of-Thought: Step-by-step generating multi-shot video with minimal manual interventioncited by 33
- 2025MultiShotMaster: A Controllable Multi-Shot Video Generation Frameworkcited by 33
- 2025InternSpatial: A Comprehensive Dataset for Spatial Reasoning in Vision-Language Modelscited by 26
- 2024E.T. the Exceptional Trajectories: Text-to-camera-trajectory generation with character awarenesscited by 24
- 2024Cinematographic Camera Diffusion Modelcited by 18
- 2025GenDoP: Auto-regressive Camera Trajectory Generation as a Director of Photographycited by 18
- 2019Autonomous Aerial Cinematography In Unstructured Environments With Learned Artistic Decision-Makingcited by 16
- 2025Cut2Next: Generating Next Shot via In-Context Tuningcited by 15
- 2025UniVA: Universal Video Agent towards Open-Source Next-Generation Video Generalistcited by 15
- 2024SplaTraj: Camera Trajectory Generation with Semantic Gaussian Splattingcited by 10
- 2025ShoulderShot: Generating Over-the-Shoulder Dialogue Videoscited by 7
- 2024Can video generation replace cinematographers? Research on the cinematic language of generated videocited by 6
50 most cited of 52 All 52 (JSON)
Built on by
Citing papers per year: 2026: 7.
7 papers.
- 2026EntityBench: Towards Entity-Consistent Long-Range Multi-Shot Video Generationcited by 6
- 2026CineDance: Towards Next-Generation Multi-Shot Long-Form Cinematic Audio-Video Generationcited by 5
- 2026Pseudo-Unification: Entropy Probing Reveals Divergent Information Patterns in Unified Multimodal Modelscited by 1
- 2026Agentic Visual Generation: From Generative Models to Agentic Controlcited by 0
- 2026Adaptive Reward Routing: Dynamic Multi-Reward Optimization for Joint Audio-Video Diffusion via Forward-Process RLcited by 0
- 2026Unveiling the Value of Motion for Cinematic Camera Trajectoriescited by 0
- 2026TKCAM: Text and Keyframe to Camera Trajectory Generationcited by 0
- 2026TKCAM: Text and Keyframe to Camera Trajectory Generation8 Oct 2026
- 2026Unveiling the Value of Motion for Cinematic Camera Trajectories30 Sep 2026
- 2026Adaptive Reward Routing: Dynamic Multi-Reward Optimization for Joint Audio-Video Diffusion via Forward-Process RL29 Sep 2026
- 2026Agentic Visual Generation: From Generative Models to Agentic Control6 Sep 2026
- 2026CineDance: Towards Next-Generation Multi-Shot Long-Form Cinematic Audio-Video Generation8 Jun 2026
- 2026EntityBench: Towards Entity-Consistent Long-Range Multi-Shot Video Generation14 May 2026
- 2026Pseudo-Unification: Entropy Probing Reveals Divergent Information Patterns in Unified Multimodal Models13 Apr 2026
Similar, not cited
Close in wording, no citation either way
- 2026CamPilot: A Multi-Agent Cinematic Assistant for Camera-Controlled Movie Generation
- 2026ShotPlan: Cinematic Video Generation with Learnable Planning Token
- 2026Camera Artist: A Multi-Agent Framework for Cinematic Language Storytelling Video Generation
- 2026CineWeaver: Training-Free Reference-Controllable Multi-Shot Long Video Generation for Cinematic Storytelling
- 2026Auteur: Language-Driven Cinematographic Framing for Human-Centric Video Generation
- 2025CineTrans: Learning to Generate Videos with Cinematic Transitions via Masked Diffusion Models
- 2026ActCam: Zero-Shot Joint Camera and 3D Motion Control for Video Generation
- 2026DreamShot: Personalized Storyboard Synthesis with Video Diffusion Prior