# GenDoP: Auto-regressive Camera Trajectory Generation as a Director of Photography

arXiv: https://arxiv.org/abs/2504.07083 · HTML: https://mlsanity.com/paper/2504.07083/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 54 references that are in the map.
- First posted on arXiv 9 Apr 2025.

## Builds on (most cited first, 30 of 36)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [HunyuanVideo: A Systematic Framework For Large Video Generative Models](https://mlsanity.com/paper/2412.03603/) (2024)
- [Scaling Autoregressive Models for Content-Rich Text-to-Image Generation](https://mlsanity.com/paper/2206.10789/) (2022)
- [Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction](https://mlsanity.com/paper/2404.02905/) (2024)
- [Cosmos World Foundation Model Platform for Physical AI](https://mlsanity.com/paper/2501.03575/) (2025)
- [VideoGPT: Video Generation using VQ-VAE and Transformers](https://mlsanity.com/paper/2104.10157/) (2021)
- [DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Vision](https://mlsanity.com/paper/2312.16256/) (2023)
- [VideoPoet: A Large Language Model for Zero-Shot Video Generation](https://mlsanity.com/paper/2312.14125/) (2023)
- [CameraCtrl: Enabling Camera Control for Text-to-Video Generation](https://mlsanity.com/paper/2404.02101/) (2024)
- [MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motion](https://mlsanity.com/paper/2410.03825/) (2024)
- [Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesis](https://mlsanity.com/paper/2412.04431/) (2024)
- [Navigation World Models](https://mlsanity.com/paper/2412.03572/) (2024)
- [MeshGPT: Generating Triangle Meshes with Decoder-Only Transformers](https://mlsanity.com/paper/2311.15475/) (2023)
- [MVImgNet: A Large-scale Dataset of Multi-view Images](https://mlsanity.com/paper/2303.06042/) (2023)
- [MeshAnything: Artist-Created Mesh Generation with Autoregressive Transformers](https://mlsanity.com/paper/2406.10163/) (2024)
- [DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusion](https://mlsanity.com/paper/2411.04928/) (2024)
- [GameGen-X: Interactive Open-world Game Video Generation](https://mlsanity.com/paper/2411.00769/) (2024)
- [Training-free Camera Control for Video Generation](https://mlsanity.com/paper/2406.10126/) (2024)
- [Loong: Generating Minute-level Long Videos with Autoregressive Language Models](https://mlsanity.com/paper/2410.02757/) (2024)
- [Voxurf: Voxel-based Efficient and Accurate Neural Surface Reconstruction](https://mlsanity.com/paper/2208.12697/) (2022)
- [EdgeRunner: Auto-regressive Auto-encoder for Artistic Mesh Generation](https://mlsanity.com/paper/2409.18114/) (2024)
- [SG-I2V: Self-Guided Trajectory Control in Image-to-Video Generation](https://mlsanity.com/paper/2411.04989/) (2024)
- [Director3D: Real-world Camera Trajectory and 3D Scene Generation from Text](https://mlsanity.com/paper/2406.17601/) (2024)
- [A Unified Framework for Shot Type Classification Based on Subject Centric Lens](https://mlsanity.com/paper/2008.03548/) (2020)
- [E.T. the Exceptional Trajectories: Text-to-camera-trajectory generation with character awareness](https://mlsanity.com/paper/2407.01516/) (2024)

## Built on by (most cited first, 18 of 18)

- [Controllable Video Generation: A Survey](https://mlsanity.com/paper/2507.16869/) (2025)
- [Pulp Motion: Framing-aware multimodal camera and human motion generation](https://mlsanity.com/paper/2510.05097/) (2025)
- [ShotVerse: Advancing Cinematic Camera Control for Text-Driven Multi-Shot Video Creation](https://mlsanity.com/paper/2603.11421/) (2026)
- [AdaViewPlanner: Adapting Video Diffusion Models for Viewpoint Planning in 4D Scenes](https://mlsanity.com/paper/2510.10670/) (2025)
- [LAMP: Language-Assisted Motion Planning for Controllable Video Generation](https://mlsanity.com/paper/2512.03619/) (2025)
- [VERTIGO: Visual Preference Optimization for Cinematic Camera Trajectory Generation](https://mlsanity.com/paper/2604.02467/) (2026)
- [Cutscene Agent: An LLM Agent Framework for Automated 3D Cutscene Generation](https://mlsanity.com/paper/2604.25318/) (2026)
- [CineLOG: A Training Free Approach for Cinematic Long Video Generation](https://mlsanity.com/paper/2512.12209/) (2025)
- [Auteur: Language-Driven Cinematographic Framing for Human-Centric Video Generation](https://mlsanity.com/paper/2606.01900/) (2026)
- [Natural Language Camera Movement Understanding](https://mlsanity.com/paper/2607.03043/) (2026)
- [CinemaTraj: Composing Atomic Camera Trajectories for 3D Scenes with LLM Agents](https://mlsanity.com/paper/2607.26910/) (2026)
- [YingVideo-MV: Music-Driven Multi-Stage Video Generation](https://mlsanity.com/paper/2512.02492/) (2025)
- [Controllable Complex Human Motion Video Generation via Text-to-Skeleton Cascades](https://mlsanity.com/paper/2603.08028/) (2026)
- [Before the Shutter: Aesthetic and Actionable Portrait Photography Planning in 3D Scenes](https://mlsanity.com/paper/2605.30318/) (2026)
- [CapFrame: Text-Instructed Viewpoint Grounding in 3D Gaussian Scenes via Geometric Pseudo Labels](https://mlsanity.com/paper/2608.30342/) (2026)
- [PACE: Precise AI Cinematic Expression](https://mlsanity.com/paper/2609.19853/) (2026)
- [AESOP: Asymmetric Human-Camera Generation with Translation-Intensity Control](https://mlsanity.com/paper/2609.37229/) (2026)
- [Unveiling the Value of Motion for Cinematic Camera Trajectories](https://mlsanity.com/paper/2609.38683/) (2026)

## Abstract (arXiv)

Camera trajectory design plays a crucial role in video production, serving as a fundamental tool for conveying directorial intent and enhancing visual storytelling. In cinematography, Directors of Photography meticulously craft camera movements to achieve expressive and intentional framing. However, existing methods for camera trajectory generation remain limited: Traditional approaches rely on geometric optimization or handcrafted procedural systems, while recent learning-based methods often inherit structural biases or lack textual alignment, constraining creative synthesis. In this work, we introduce an auto-regressive model inspired by the expertise of Directors of Photography to generate artistic and expressive camera trajectories. We first introduce DataDoP, a large-scale multi-modal dataset containing 29K real-world shots with free-moving camera trajectories, depth maps, and detailed captions in specific movements, interaction with the scene, and directorial intent. Thanks to the comprehensive and diverse database, we further train an auto-regressive, decoder-only Transformer for high-quality, context-aware camera movement generation based on text guidance and RGBD inputs, named GenDoP. Extensive experiments demonstrate that compared to existing methods, GenDoP offers better controllability, finer-grained trajectory adjustments, and higher motion stability. We believe our approach establishes a new standard for learning-based cinematography, paving the way for future advancements in camera control and filmmaking. Our project website: https://kszpxxzmc.github.io/GenDoP/.