Captain Safari: A World Engine with Pose-Aligned 3D Memory
Not on the map. Why?
Abstract (arXiv)
World engines aim to synthesize long, 3D-consistent videos that support interactive exploration of a scene under user-controlled camera motion. However, existing systems struggle under aggressive 6-DoF trajectories and complex outdoor layouts: they lose long-range geometric coherence, deviate from the target path, or collapse into overly conservative motion. To this end, we introduce Captain Safari, a pose-conditioned world engine that generates videos by retrieving from a persistent world memory. Given a camera path, our method maintains a dynamic local memory and uses a retriever to fetch pose-aligned world tokens, which then condition video generation along the trajectory. This design enables the model to maintain stable 3D structure while accurately executing challenging camera maneuvers. To evaluate this setting, we curate OpenSafari, a new in-the-wild FPV dataset containing high-dynamic drone videos with verified camera trajectories, constructed through a multi-stage geometric and kinematic validation pipeline. Across video quality, 3D consistency, and trajectory following, Captain Safari substantially outperforms state-of-the-art camera-controlled generators. It reduces MEt3R from 0.3703 to 0.3690, improves AUC@30 from 0.181 to 0.200, and yields substantially lower FVD than all camera-controlled baselines. More importantly, in a 50-participant, 5-way human study where annotators select the best result among five anonymized models, 67.6% of preferences favor our method across all axes. Our results demonstrate that pose-conditioned world memory is a powerful mechanism for long-horizon, controllable video generation and provide OpenSafari as a challenging new benchmark for future world-engine research.
Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 55 of its 68 references that are in the map. First posted on arXiv 28 Nov 2025.
Builds on
55 of 68 references.
References · 55
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
- 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
- 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,584
- 2025VGGT: Visual Geometry Grounded Transformercited by 1,539
- 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,489
- 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,443
- 2024Depth Anything V2cited by 1,422
- 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
- 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,234
- 2019Habitat: A Platform for Embodied AI Researchcited by 1,191
38 more
- 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 592
- 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 580
- 2018From Coarse to Fine: Robust Hierarchical Localization at Large Scalecited by 559
- 2023VideoComposer: Compositional Video Synthesis with Motion Controllabilitycited by 464
- 2024CameraCtrl: Enabling Camera Control for Text-to-Video Generationcited by 460
- 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesiscited by 403
- 2019SynSin: End-to-end View Synthesis from a Single Imagecited by 364
- 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videoscited by 345
- 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 341
- 2024WonderWorld: Interactive 3D Scene Generation from a Single Imagecited by 202
- 2024Motion Prompting: Controlling Video Generation with Motion Trajectoriescited by 178
- 2024CamCo: Camera-Controllable 3D-Consistent Image-to-Video Generationcited by 166
- 2024Tora: Trajectory-oriented Diffusion Transformer for Video Generationcited by 164
- 2025Streaming 4D Visual Geometry Transformercited by 161
- 2025Video World Models with Long-term Spatial Memorycited by 136
- 2024Training-free Camera Control for Video Generationcited by 122
- 2024CamI2V: Camera-Controlled Image-to-Video Diffusion Modelcited by 120
- 2024Collaborative Video Diffusion: Consistent Multi-video Generation with Camera Controlcited by 117
- 2025AdaWorld: Learning Adaptable World Models with Latent Actionscited by 114
- 2025Voyager: Long-Range and World-Consistent Video Diffusion for Explorable 3D Scene Generationcited by 101
- 2024Wonderland: Navigating 3D Scenes from a Single Imagecited by 99
- 2025Geometry Forcing: Marrying Video Diffusion and 3D Representation for Consistent World Modelingcited by 96
- 2025MEt3R: Measuring Multi-View Consistency in Generated Imagescited by 95
- 2023WoVoGen: World Volume-aware Diffusion for Controllable Multi-camera Driving Scene Generationcited by 93
- 2021SceneGen: Learning to Generate Realistic Traffic Scenescited by 84
- 2020Worldsheet: Wrapping the World in a 3D Sheet for View Synthesis from a Single Imagecited by 77
- 2024TC4D: Trajectory-Conditioned Text-to-4D Generationcited by 77
- 2024ReVideo: Remake a Video with Motion and Content Controlcited by 74
- 2025RealCam-I2V: Real-World Image-to-Video Generation with Interactive Complex Camera Controlcited by 57
- 2025Memory Forcing: Spatio-Temporal Memory for Consistent Scene Generation on Minecraftcited by 52
- 2025VMoBA: Mixture-of-Block Attention for Video Diffusion Modelscited by 37
- 2024SimGen: Simulator-conditioned Driving Scene Generationcited by 34
- 2024Baking Gaussian Splatting into Diffusion Denoiser for Fast and Scalable Single-stage Image-to-3D Generation and Reconstructioncited by 32
- 2025WonderTurbo: Generating Interactive 3D World in 0.72 Secondscited by 32
- 2024Video Diffusion Models: A Surveycited by 30
- 2025MotionPro: A Precise Motion Controller for Image-to-Video Generationcited by 30
- 2025WorldExplorer: Towards Generating Fully Navigable 3D Scenescited by 26
- 2024Cavia: Camera-controllable Multi-view Video Diffusion with View-Integrated Attentioncited by 24
50 most cited of 55 All 55 (JSON)
Built on by
Citing papers per year: 2026: 5.
5 papers.
- 2026Out of Sight but Not Out of Mind: Hybrid Memory for Dynamic Video World Modelscited by 26
- 2026AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memoriescited by 15
- 2026Prisma-World: Camera-Controllable Multi-Agent Video World Modelcited by 11
- 2026Evolution of Video Generative Foundationscited by 7
- 2026PanoWorld: Real-World Panoramic Generationcited by 5
- 2026PanoWorld: Real-World Panoramic Generation10 Jul 2026
- 2026Prisma-World: Camera-Controllable Multi-Agent Video World Model8 Jun 2026
- 2026Evolution of Video Generative Foundations7 Apr 2026
- 2026Out of Sight but Not Out of Mind: Hybrid Memory for Dynamic Video World Models26 Mar 2026
- 2026AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories16 Feb 2026