# UCM: Unified Modeling of Camera Control and Memory with Time-aware Positional Encoding Warping for World Models

arXiv: https://arxiv.org/abs/2602.22960 · HTML: https://mlsanity.com/paper/2602.22960/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Video-generation world models and simulators, one of 11 directions in World models.
- Builds on 39 of its 65 references that are in the map.
- First posted on arXiv 26 Feb 2026.

## Placed in

- World models / [Video-generation world models and simulators](https://mlsanity.com/wm/video-generation-world-models-and-simulators/)

## Builds on (most cited first, 30 of 39)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [HunyuanVideo: A Systematic Framework For Large Video Generative Models](https://mlsanity.com/paper/2412.03603/) (2024)
- [Towards Accurate Generative Models of Video: A New Metric & Challenges](https://mlsanity.com/paper/1812.01717/) (2018)
- [Depth Anything 3: Recovering the Visual Space from Any Views](https://mlsanity.com/paper/2511.10647/) (2025)
- [Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion](https://mlsanity.com/paper/2407.01392/) (2024)
- [Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion](https://mlsanity.com/paper/2506.08009/) (2025)
- [MotionCtrl: A Unified and Flexible Motion Controller for Video Generation](https://mlsanity.com/paper/2312.03641/) (2023)
- [DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Vision](https://mlsanity.com/paper/2312.16256/) (2023)
- [3DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructions](https://mlsanity.com/paper/1603.08182/) (2016)
- [Diffusion for World Modeling: Visual Details Matter in Atari](https://mlsanity.com/paper/2405.12399/) (2024)
- [Diffusion Models Are Real-Time Game Engines](https://mlsanity.com/paper/2408.14837/) (2024)
- [Diffusion as Shader: 3D-aware Video Diffusion for Versatile Video Generation Control](https://mlsanity.com/paper/2501.03847/) (2025)
- [WorldMem: Long-term Consistent World Simulation with Memory](https://mlsanity.com/paper/2504.12369/) (2025)
- [Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrieval](https://mlsanity.com/paper/2506.03141/) (2025)
- [AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformers](https://mlsanity.com/paper/2411.18673/) (2024)
- [MiraData: A Large-Scale Video Dataset with Long Durations and Structured Captions](https://mlsanity.com/paper/2407.06358/) (2024)
- [Video World Models with Long-term Spatial Memory](https://mlsanity.com/paper/2506.05284/) (2025)
- [Recent Advances in 3D Gaussian Splatting](https://mlsanity.com/paper/2403.11134/) (2024)
- [GameGen-X: Interactive Open-world Game Video Generation](https://mlsanity.com/paper/2411.00769/) (2024)
- [Long-Context Autoregressive Video Modeling with Next-Frame Prediction](https://mlsanity.com/paper/2503.19325/) (2025)
- [Go-with-the-Flow: Motion-Controllable Video Diffusion Models Using Real-Time Warped Noise](https://mlsanity.com/paper/2501.08331/) (2025)
- [Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyond](https://mlsanity.com/paper/2405.03520/) (2024)
- [VMem: Consistent Interactive Video Scene Generation with Surfel-Indexed View Memory](https://mlsanity.com/paper/2506.18903/) (2025)
- [STream3R: Scalable Sequential 3D Reconstruction with Causal Transformer](https://mlsanity.com/paper/2508.10893/) (2025)
- [NVS-Solver: Video Diffusion Model as Zero-Shot Novel View Synthesizer](https://mlsanity.com/paper/2405.15364/) (2024)

## Built on by (most cited first, 9 of 9)

- [SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformer](https://mlsanity.com/paper/2605.15178/) (2026)
- [Geometry-Aware Implicit Memory for Video World Models](https://mlsanity.com/paper/2606.02436/) (2026)
- [Echo-Memory: A Controlled Study of Memory in Action World Models](https://mlsanity.com/paper/2606.09803/) (2026)
- [SCoPE: Sightline-Coordinate Positional Encoding for Video Diffusion Transformers](https://mlsanity.com/paper/2606.27345/) (2026)
- [Real-Time Human-Centric World Modeling for Upper-Body Human-Object Interaction](https://mlsanity.com/paper/2607.23517/) (2026)
- [Geometry as Address: Routing Attention to Visual Memory for Long-Horizon Camera-Controlled Video Generation](https://mlsanity.com/paper/2609.34722/) (2026)
- [Honeycomb: Constant-Size Scene Memory Representation for Video World Models](https://mlsanity.com/paper/2609.37690/) (2026)
- [GLIMPSE: Global-Local Indexing with Multi-Precision Spectral Encoding for Explorable Video World Models](https://mlsanity.com/paper/2610.13706/) (2026)
- [World-T3: Test-Time Training as Persistent Memory for Real-Time Video World Models](https://mlsanity.com/paper/2610.17434/) (2026)

## Abstract (arXiv)

World models based on video generation demonstrate remarkable potential for simulating interactive environments yet suffer from persistent difficulties in two key areas: maintaining long-term content consistency when scenes are revisited and enabling precise camera control from user-specified inputs. Existing methods based on explicit 3D reconstruction often compromise flexibility in unbounded scenarios and struggle to preserve fine-grained structures. Alternative methods rely directly on previously generated frames without establishing explicit spatial correspondence, thereby limiting controllability and consistency. To address these limitations, we present UCM, a novel framework for unified modeling of long-term memory and precise camera control via a time-aware positional encoding warping mechanism. To reduce computational overhead, we design an efficient dual-stream diffusion transformer for high-fidelity generation. Moreover, we introduce a scalable data curation strategy that utilizes point-cloud-based rendering to simulate scene revisiting, enabling training on over 500K monocular videos. Extensive experiments on real-world and synthetic benchmarks demonstrate that UCM significantly outperforms state-of-the-art methods on long-term scene consistency, while achieving precise camera controllability in high-fidelity video generation.