# Unified Camera Positional Encoding for Controlled Video Generation

arXiv: https://arxiv.org/abs/2512.07237 · HTML: https://mlsanity.com/paper/2512.07237/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Placed in Video-generation world models and simulators, one of 11 directions in World models.
- Builds on 20 of its 77 references that are in the map.
- First posted on arXiv 8 Dec 2025.

## Placed in

- World models / [Video-generation world models and simulators](https://mlsanity.com/wm/video-generation-world-models-and-simulators/)

## Builds on (most cited first, 20 of 20)

- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [HunyuanVideo: A Systematic Framework For Large Video Generative Models](https://mlsanity.com/paper/2412.03603/) (2024)
- [Towards Accurate Generative Models of Video: A New Metric & Challenges](https://mlsanity.com/paper/1812.01717/) (2018)
- [CameraCtrl: Enabling Camera Control for Text-to-Video Generation](https://mlsanity.com/paper/2404.02101/) (2024)
- [Seedance 1.0: Exploring the Boundaries of Video Generation Models](https://mlsanity.com/paper/2506.09113/) (2025)
- [Stable Virtual Camera: Generative View Synthesis with Diffusion Models](https://mlsanity.com/paper/2503.14489/) (2025)
- [ViPE: Video Pose Engine for 3D Geometric Perception](https://mlsanity.com/paper/2508.10934/) (2025)
- [CamI2V: Camera-Controlled Image-to-Video Diffusion Model](https://mlsanity.com/paper/2410.15957/) (2024)
- [Cameras as Relative Positional Encoding](https://mlsanity.com/paper/2507.10496/) (2025)
- [CameraCtrl II: Dynamic Scene Exploration via Camera-controlled Video Diffusion Models](https://mlsanity.com/paper/2503.10592/) (2025)
- [HunyuanWorld 1.0: Generating Immersive, Explorable, and Interactive 3D Worlds from Words or Pixels](https://mlsanity.com/paper/2507.21809/) (2025)
- [RayZer: A Self-supervised Large View Synthesis Model](https://mlsanity.com/paper/2505.00702/) (2025)
- [Matrix-3D: Omnidirectional Explorable 3D World Generation](https://mlsanity.com/paper/2508.08086/) (2025)
- [PanoWan: Lifting Diffusion Video Generation Models to 360deg with Latitude/Longitude-aware Mechanisms](https://mlsanity.com/paper/2505.22016/) (2025)
- [Boosting Camera Motion Control for Video Diffusion Transformers](https://mlsanity.com/paper/2410.10802/) (2024)
- [Zero-Shot Novel View and Depth Synthesis with Multi-View Geometric Diffusion](https://mlsanity.com/paper/2501.18804/) (2025)
- [I2V3D: Controllable image-to-video generation with 3D guidance](https://mlsanity.com/paper/2503.09733/) (2025)
- [RePAST: Relative Pose Attention Scene Representation Transformer](https://mlsanity.com/paper/2304.00947/) (2023)

## Built on by (most cited first, 30 of 37)

- [SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformer](https://mlsanity.com/paper/2605.15178/) (2026)
- [AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories](https://mlsanity.com/paper/2602.14941/) (2026)
- [RayRoPE: Projective Ray Positional Encoding for Multi-view Attention](https://mlsanity.com/paper/2601.15275/) (2026)
- [ReRoPE: Repurposing RoPE for Relative Camera Control](https://mlsanity.com/paper/2602.08068/) (2026)
- [EchoWM: Open and Enterable Omnimodal World Models](https://mlsanity.com/paper/2608.23189/) (2026)
- [Prisma-World: Camera-Controllable Multi-Agent Video World Model](https://mlsanity.com/paper/2606.09507/) (2026)
- [UCM: Unified Modeling of Camera Control and Memory with Time-aware Positional Encoding Warping for World Models](https://mlsanity.com/paper/2602.22960/) (2026)
- [URoPE: Universal Relative Position Embedding across Geometric Spaces](https://mlsanity.com/paper/2604.18747/) (2026)
- [BiWM: Advancing Open-Source Interactive Video World Models with Bidirectional Autoregression](https://mlsanity.com/paper/2606.10135/) (2026)
- [Compression and Retrieval: Implicit Memory Retrieval for Video World Models](https://mlsanity.com/paper/2606.23105/) (2026)
- [AlayaWorld: Long-Horizon and Playable Video World Generation](https://mlsanity.com/paper/2607.06291/) (2026)
- [SCoPE: Sightline-Coordinate Positional Encoding for Video Diffusion Transformers](https://mlsanity.com/paper/2606.27345/) (2026)
- [PanoWorld: Real-World Panoramic Generation](https://mlsanity.com/paper/2607.09661/) (2026)
- [CRePE: Curved Ray Expectation Positional Encoding for Unified-Camera-Controlled Video Generation](https://mlsanity.com/paper/2605.12938/) (2026)
- [Warp-as-History: Generalizable Camera-Controlled Video Generation from One Training Video](https://mlsanity.com/paper/2605.15182/) (2026)
- [Teaching Video Generators to Remember: Eliciting Dynamic Memory for Out-of-Sight State Evolution](https://mlsanity.com/paper/2605.25333/) (2026)
- [From Pixels to States: Rethinking Interactive World Models as Game Engines](https://mlsanity.com/paper/2607.14076/) (2026)
- [Long-Horizon Audio-Visual Generation for Persistent Stories and Interactive Worlds](https://mlsanity.com/paper/2608.23383/) (2026)
- [RAWild: Sensor-Agnostic RAW Object Detection via Physics-Guided Curve and Grid Modeling](https://mlsanity.com/paper/2605.05941/) (2026)
- [GimbalDiffusion: Gravity-Aware Camera Control for Video Generation](https://mlsanity.com/paper/2512.09112/) (2025)
- [I3DM: Implicit 3D-aware Memory Retrieval and Injection for Consistent Video Scene Generation](https://mlsanity.com/paper/2603.23413/) (2026)
- [CalibAnyView: Beyond Single-View Camera Calibration in the Wild](https://mlsanity.com/paper/2605.14615/) (2026)
- [Grounded-Exo2Ego: Structured Semantic Grounding for Robust Exocentric-to-Egocentric Video Generation](https://mlsanity.com/paper/2608.20534/) (2026)
- [DPPE: Rethinking Camera-Based Positional Encoding for Scaling Multi-View Transformers](https://mlsanity.com/paper/2606.31585/) (2026)
- [CameraAnything: Refilming Videos with Arbitrary Camera Control](https://mlsanity.com/paper/2607.24591/) (2026)
- [MeRoPE: Metric Rotary Position Embedding for Camera-Controlled Video Generation](https://mlsanity.com/paper/2609.01252/) (2026)
- [G-ray: Ray-Level Relative Geometric Position Encoding in Multi-View Vision Transformers under Camera Heterogeneity](https://mlsanity.com/paper/2609.15018/) (2026)
- [Geometrically Consistent Multi-View Scene Generation from Freehand Sketches](https://mlsanity.com/paper/2604.14302/) (2026)
- [Effective Multi-sensor Conditioning for Street-view Novel-view Synthesis](https://mlsanity.com/paper/2606.01590/) (2026)
- [Flex4DHuman: Flexible Multi-view Video Diffusion for 4D Human Reconstruction](https://mlsanity.com/paper/2606.13655/) (2026)

## Abstract (arXiv)

Transformers have emerged as a universal backbone across 3D perception, video generation, and world models for autonomous driving and embodied AI, where understanding camera geometry is essential for grounding visual observations in three-dimensional space. However, existing camera encoding methods often rely on simplified pinhole assumptions, restricting generalization across the diverse intrinsics and lens distortions in real-world cameras. We introduce Relative Ray Encoding, a geometry-consistent representation that unifies complete camera information, including 6-DoF poses, intrinsics, and lens distortions. To evaluate its capability under diverse controllability demands, we adopt camera-controlled text-to-video generation as a testbed task. Within this setting, we further identify pitch and roll as two components effective for Absolute Orientation Encoding, enabling full control over the initial camera orientation. Together, these designs form UCPE (Unified Camera Positional Encoding), which integrates into a pretrained video Diffusion Transformer through a lightweight spatial attention adapter, adding less than 1% trainable parameters while achieving state-of-the-art camera controllability and visual fidelity. To facilitate systematic training and evaluation, we construct a large video dataset covering a wide range of camera motions and lens types. Extensive experiments validate the effectiveness of UCPE in camera-controllable video generation and highlight its potential as a general camera representation for Transformers across future multi-view, video, and 3D tasks. Code will be available at https://github.com/chengzhag/UCPE.