UCM: Unified Modeling of Camera Control and Memory with Time-aware Positional Encoding Warping for World Models
On the map: World models → Video-generation world models and simulators
Abstract (arXiv)
World models based on video generation demonstrate remarkable potential for simulating interactive environments yet suffer from persistent difficulties in two key areas: maintaining long-term content consistency when scenes are revisited and enabling precise camera control from user-specified inputs. Existing methods based on explicit 3D reconstruction often compromise flexibility in unbounded scenarios and struggle to preserve fine-grained structures. Alternative methods rely directly on previously generated frames without establishing explicit spatial correspondence, thereby limiting controllability and consistency. To address these limitations, we present UCM, a novel framework for unified modeling of long-term memory and precise camera control via a time-aware positional encoding warping mechanism. To reduce computational overhead, we design an efficient dual-stream diffusion transformer for high-fidelity generation. Moreover, we introduce a scalable data curation strategy that utilizes point-cloud-based rendering to simulate scene revisiting, enabling training on over 500K monocular videos. Extensive experiments on real-world and synthetic benchmarks demonstrate that UCM significantly outperforms state-of-the-art methods on long-term scene consistency, while achieving precise camera controllability in high-fidelity video generation.
Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 39 of its 65 references that are in the map. First posted on arXiv 26 Feb 2026.
Builds on
39 of 65 references.
Closest ancestors · 7
Same direction, newest first
- 2025Unified Camera Positional Encoding for Controlled Video Generation8 Dec
- 2025Video World Models with Long-term Spatial Memory5 Jun
- 2025WorldMem: Long-term Consistent World Simulation with Memory16 Apr
- 2024GameGen-X: Interactive Open-world Game Video Generation1 Nov
- 2024Diffusion Models Are Real-Time Game Engines27 Aug
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer12 Aug
- 2024Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyond6 May
Other references · 32
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
- 2022Classifier-Free Diffusion Guidancecited by 5,717
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
- 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
- 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
- 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
- 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 634
- 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 626
- 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 569
- 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 568
- 2023DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Visioncited by 514
20 more
- 20163DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructionscited by 333
- 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 285
- 2025Diffusion as Shader: 3D-aware Video Diffusion for Versatile Video Generation Controlcited by 184
- 2025Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrievalcited by 157
- 2024AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformerscited by 151
- 2024MiraData: A Large-Scale Video Dataset with Long Durations and Structured Captionscited by 144
- 2024Recent Advances in 3D Gaussian Splattingcited by 130
- 2025Long-Context Autoregressive Video Modeling with Next-Frame Predictioncited by 119
- 2025Go-with-the-Flow: Motion-Controllable Video Diffusion Models Using Real-Time Warped Noisecited by 114
- 2025VMem: Consistent Interactive Video Scene Generation with Surfel-Indexed View Memorycited by 102
- 2024NVS-Solver: Video Diffusion Model as Zero-Shot Novel View Synthesizercited by 79
- 2025STream3R: Scalable Sequential 3D Reconstruction with Causal Transformercited by 76
- 2025Uni3C: Unifying Precisely 3D-Enhanced Camera and Human Motion Controls for Video Generationcited by 75
- 2025SpatialVID: A Large-Scale Video Dataset with Spatial Annotationscited by 72
- 2025WorldMirror: Universal 3D World Reconstruction with Any-Prior Promptingcited by 60
- 2025RealCam-I2V: Real-World Image-to-Video Generation with Interactive Complex Camera Controlcited by 54
- 2024I2VControl-Camera: Precise Video Camera Control with Adjustable Motion Strengthcited by 38
- 2025StarGen: A Spatiotemporal Autoregression Framework with Video Diffusion Model for Scalable and Controllable Scene Generationcited by 26
- 2025Genesis: Multimodal Driving Scene Generation with Spatio-Temporal and Cross-Modal Consistencycited by 21
- 2025Towards foundational LiDAR world models with efficient latent flow matchingcited by 10
Built on by
Citing papers per year: 2026: 7.
7 papers.
- 2026SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformercited by 50
- 2026Geometry-Aware Implicit Memory for Video World Modelscited by 7
- 2026Echo-Memory: A Controlled Study of Memory in Action World Modelscited by 4
- 2026SCoPE: Sightline-Coordinate Positional Encoding for Video Diffusion Transformerscited by 3
- 2026Real-Time Human-Centric World Modeling for Upper-Body Human-Object Interactioncited by 0
- 2026Geometry as Address: Routing Attention to Visual Memory for Long-Horizon Camera-Controlled Video Generationcited by 0
- 2026Honeycomb: Constant-Size Scene Memory Representation for Video World Modelscited by 0
- 2026Honeycomb: Constant-Size Scene Memory Representation for Video World Models29 Sep 2026
- 2026Geometry as Address: Routing Attention to Visual Memory for Long-Horizon Camera-Controlled Video Generation28 Sep 2026
- 2026Real-Time Human-Centric World Modeling for Upper-Body Human-Object Interaction26 Jul 2026
- 2026SCoPE: Sightline-Coordinate Positional Encoding for Video Diffusion Transformers25 Jun 2026
- 2026Echo-Memory: A Controlled Study of Memory in Action World Models8 Jun 2026
- 2026Geometry-Aware Implicit Memory for Video World Models1 Jun 2026
- 2026SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformer14 May 2026
Similar, not cited
Close in wording, no citation either way
- 2026WorldStereo: Bridging Camera-Guided Video Generation and Scene Reconstruction via 3D Geometric Memories
- 2024VD3D: Taming Large Video Diffusion Transformers for 3D Camera Control
- 2026WorldCrafter: Consistent Video World Model with Implicit 3D-aware Memory
- 2026Beyond Pixel Histories: World Models with Persistent 3D State
- 2025WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modeling
- 2025Voyager: Long-Range and World-Consistent Video Diffusion for Explorable 3D Scene Generation
- 2026PE-Field 4D: Video Generation Models as Canvas
- 2026GeoStream: Toward Precise Camera Controlled Streaming Video Generation