mlsanityData to 14 Oct 2026

Captain Safari: A World Engine with Pose-Aligned 3D Memory

arXiv 2511.22815 ↗First posted 28 Nov 2025cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

World engines aim to synthesize long, 3D-consistent videos that support interactive exploration of a scene under user-controlled camera motion. However, existing systems struggle under aggressive 6-DoF trajectories and complex outdoor layouts: they lose long-range geometric coherence, deviate from the target path, or collapse into overly conservative motion. To this end, we introduce Captain Safari, a pose-conditioned world engine that generates videos by retrieving from a persistent world memory. Given a camera path, our method maintains a dynamic local memory and uses a retriever to fetch pose-aligned world tokens, which then condition video generation along the trajectory. This design enables the model to maintain stable 3D structure while accurately executing challenging camera maneuvers. To evaluate this setting, we curate OpenSafari, a new in-the-wild FPV dataset containing high-dynamic drone videos with verified camera trajectories, constructed through a multi-stage geometric and kinematic validation pipeline. Across video quality, 3D consistency, and trajectory following, Captain Safari substantially outperforms state-of-the-art camera-controlled generators. It reduces MEt3R from 0.3703 to 0.3690, improves AUC@30 from 0.181 to 0.200, and yields substantially lower FVD than all camera-controlled baselines. More importantly, in a 50-participant, 5-way human study where annotators select the best result among five anonymized models, 67.6% of preferences favor our method across all axes. Our results demonstrate that pose-conditioned world memory is a powerful mechanism for long-horizon, controllable video generation and provide OpenSafari as a challenging new benchmark for future world-engine research.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 55 of its 68 references that are in the map. First posted on arXiv 28 Nov 2025.

Builds on

55 of 68 references.

References · 55

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  2. 2025Qwen2.5-VL Technical Reportcited by 5,587
  3. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  4. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
  5. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,584
  6. 2025VGGT: Visual Geometry Grounded Transformercited by 1,539
  7. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,489
  8. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,443
  9. 2024Depth Anything V2cited by 1,422
  10. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
  11. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,234
  12. 2019Habitat: A Platform for Embodied AI Researchcited by 1,191
38 more
  1. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 592
  2. 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 580
  3. 2018From Coarse to Fine: Robust Hierarchical Localization at Large Scalecited by 559
  4. 2023VideoComposer: Compositional Video Synthesis with Motion Controllabilitycited by 464
  5. 2024CameraCtrl: Enabling Camera Control for Text-to-Video Generationcited by 460
  6. 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesiscited by 403
  7. 2019SynSin: End-to-end View Synthesis from a Single Imagecited by 364
  8. 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videoscited by 345
  9. 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 341
  10. 2024WonderWorld: Interactive 3D Scene Generation from a Single Imagecited by 202
  11. 2024Motion Prompting: Controlling Video Generation with Motion Trajectoriescited by 178
  12. 2024CamCo: Camera-Controllable 3D-Consistent Image-to-Video Generationcited by 166
  13. 2024Tora: Trajectory-oriented Diffusion Transformer for Video Generationcited by 164
  14. 2025Streaming 4D Visual Geometry Transformercited by 161
  15. 2025Video World Models with Long-term Spatial Memorycited by 136
  16. 2024Training-free Camera Control for Video Generationcited by 122
  17. 2024CamI2V: Camera-Controlled Image-to-Video Diffusion Modelcited by 120
  18. 2024Collaborative Video Diffusion: Consistent Multi-video Generation with Camera Controlcited by 117
  19. 2025AdaWorld: Learning Adaptable World Models with Latent Actionscited by 114
  20. 2025Voyager: Long-Range and World-Consistent Video Diffusion for Explorable 3D Scene Generationcited by 101
  21. 2024Wonderland: Navigating 3D Scenes from a Single Imagecited by 99
  22. 2025Geometry Forcing: Marrying Video Diffusion and 3D Representation for Consistent World Modelingcited by 96
  23. 2025MEt3R: Measuring Multi-View Consistency in Generated Imagescited by 95
  24. 2023WoVoGen: World Volume-aware Diffusion for Controllable Multi-camera Driving Scene Generationcited by 93
  25. 2021SceneGen: Learning to Generate Realistic Traffic Scenescited by 84
  26. 2020Worldsheet: Wrapping the World in a 3D Sheet for View Synthesis from a Single Imagecited by 77
  27. 2024TC4D: Trajectory-Conditioned Text-to-4D Generationcited by 77
  28. 2024ReVideo: Remake a Video with Motion and Content Controlcited by 74
  29. 2025RealCam-I2V: Real-World Image-to-Video Generation with Interactive Complex Camera Controlcited by 57
  30. 2025Memory Forcing: Spatio-Temporal Memory for Consistent Scene Generation on Minecraftcited by 52
  31. 2025VMoBA: Mixture-of-Block Attention for Video Diffusion Modelscited by 37
  32. 2024SimGen: Simulator-conditioned Driving Scene Generationcited by 34
  33. 2024Baking Gaussian Splatting into Diffusion Denoiser for Fast and Scalable Single-stage Image-to-3D Generation and Reconstructioncited by 32
  34. 2025WonderTurbo: Generating Interactive 3D World in 0.72 Secondscited by 32
  35. 2024Video Diffusion Models: A Surveycited by 30
  36. 2025MotionPro: A Precise Motion Controller for Image-to-Video Generationcited by 30
  37. 2025WorldExplorer: Towards Generating Fully Navigable 3D Scenescited by 26
  38. 2024Cavia: Camera-controllable Multi-view Video Diffusion with View-Integrated Attentioncited by 24

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 5.

5 papers.