VERTIGO: Visual Preference Optimization for Cinematic Camera Trajectory Generation
Not on the map. Why?
Abstract (arXiv)
Cinematic camera control relies on a tight feedback loop between director and cinematographer, where camera motion and framing are continuously reviewed and refined. Recent generative camera systems can produce diverse, text-conditioned trajectories, but they lack this "director in the loop" and have no explicit supervision of whether a shot is visually desirable. This results in in-distribution camera motion but poor framing, off-screen characters, and undesirable visual aesthetics. In this paper, we introduce VERTIGO, the first framework for visual preference optimization of camera trajectory generators. Our framework leverages a real-time graphics engine (Unity) to render 2D visual previews from generated camera motion. A cinematically fine-tuned vision-language model then scores these previews using our proposed cyclic semantic similarity mechanism, which aligns renders with text prompts. This process provides the visual preference signals for Direct Preference Optimization (DPO) post-training. Both quantitative evaluations and user studies on Unity renders and diffusion-based Camera-to-Video pipelines show consistent gains in condition adherence, framing quality, and perceptual realism. Notably, VERTIGO reduces the character off-screen rate from 38% to nearly 0% while preserving the geometric fidelity of camera motion. User study participants further prefer VERTIGO over baselines across composition, consistency, prompt adherence, and aesthetic quality, confirming the perceptual benefits of our visual preference post-training.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 62 of its 85 references that are in the map. First posted on arXiv 2 Apr 2026.
Builds on
62 of 85 references.
References · 62
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
- 2022Classifier-Free Diffusion Guidancecited by 5,717
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
- 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
- 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
- 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
- 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
- 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
- 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,186
- 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,086
- 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,047
38 more
- 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 568
- 2025VACE: All-in-One Video Creation and Editingcited by 467
- 2024CameraCtrl: Enabling Camera Control for Text-to-Video Generationcited by 448
- 2024Multilingual E5 Text Embeddings: A Technical Reportcited by 357
- 2023EvalCrafter: Benchmarking and Evaluating Large Video Generation Modelscited by 332
- 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 325
- 2025Improving Video Generation with Human Feedbackcited by 267
- 2025Video-MMMU: Evaluating Knowledge Acquisition from Multi-Discipline Professional Videoscited by 225
- 2024Direct-a-Video: Customized Video Generation with User-Directed Camera Movement and Object Motioncited by 167
- 2024Direct Preference Optimization of Video Large Multimodal Models from Language Model Rewardcited by 145
- 2023Rich Human Feedback for Text-to-Image Generationcited by 139
- 2024Collaborative Video Diffusion: Consistent Multi-video Generation with Camera Controlcited by 117
- 2024VideoDPO: Omni-Preference Alignment for Video Diffusion Generationcited by 102
- 2024Video Diffusion Alignment via Reward Gradientscited by 101
- 2025Self-Rewarding Vision-Language Model via Reasoning Decompositioncited by 95
- 2025CineMaster: A 3D-Aware and Controllable Framework for Cinematic Text-to-Video Generationcited by 73
- 2024T2V-Turbo-v2: Enhancing Video Generation Model Post-Training through Data, Reward, and Conditional Guidance Designcited by 54
- 2024MotionMaster: Training-free Camera Motion Transfer For Video Generationcited by 50
- 2025MotionCanvas: Cinematic Shot Design with Controllable Image-to-Video Generationcited by 50
- 2023Control-A-Video: Controllable Text-to-Video Diffusion Models with Motion Prior and Reward Feedback Learningcited by 46
- 2024Improving Dynamic Object Interactions in Text-to-Video Generation with AI Feedbackcited by 45
- 2025NewtonGen: Physics-Consistent and Controllable Text-to-Video Generation via Neural Newtonian Dynamicscited by 40
- 2024Tuning Large Multimodal Models for Videos using Reinforcement Learning from AI Feedbackcited by 36
- 2025Delving into RL for Image Generation with CoT: A Study on DPO vs. GRPOcited by 31
- 2023Enhancing Diffusion Models with Text-Encoder Reinforcement Learningcited by 30
- 2024E.T. the Exceptional Trajectories: Text-to-camera-trajectory generation with character awarenesscited by 24
- 2025Cycle Consistency as Reward: Learning Image-Text Alignment without Human Preferencescited by 22
- 2025ShotBench: Expert-Level Cinematic Understanding in Vision-Language Modelscited by 22
- 2023Dynamic Storyboard Generation in an Engine-based Virtual Environment for Video Productioncited by 21
- 2024Cinematographic Camera Diffusion Modelcited by 18
- 2024PatchDPO: Patch-level DPO for Finetuning-free Personalized Image Generationcited by 18
- 2025GenDoP: Auto-regressive Camera Trajectory Generation as a Director of Photographycited by 18
- 2024AKiRa: Augmentation Kit on Rays for optical video generationcited by 17
- 2019Autonomous Aerial Cinematography In Unstructured Environments With Learned Artistic Decision-Makingcited by 16
- 2025Sailing by the Stars: A Survey on Reward Models and Learning Strategies for Learning from Rewardscited by 14
- 2023JAWS: Just A Wild Shot for Cinematic Transfer in Neural Radiance Fieldscited by 13
- 2024Regression-aware Inference with LLMscited by 13
- 2023Cinematic Behavior Transfer via NeRF-based Differentiable Filmingcited by 12
50 most cited of 62 All 62 (JSON)
Built on by
Citing papers per year: 2026: 3.
3 papers.
Similar, not cited
Close in wording, no citation either way
- 2024Can video generation replace cinematographers? Research on the cinematic language of generated video
- 2026Vertigo Vertigo: Reconstructing a Cinematic Ideal through its Predictive AI Double
- 2025VMBench: A Benchmark for Perception-Aligned Video Motion Generation
- 2024DreamCinema: Cinematic Transfer with Free Camera and 3D Character
- 2025Stable Cinemetrics : Structured Taxonomy and Evaluation for Professional Video Generation
- 2025CineTechBench: A Benchmark for Cinematographic Technique Understanding and Generation
- 2026AesRM: Improving Video Aesthetics with Expert-Level Feedback
- 2026ActCam: Zero-Shot Joint Camera and 3D Motion Control for Video Generation