mlsanityData to 14 Oct 2026

ViPE: Video Pose Engine for 3D Geometric Perception

arXiv 2508.10934 ↗First posted 12 Aug 2025cs.CV cs.GR cs.ROcited by 170

Not on the map. Why?

Cited by 51 in World models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Accurate 3D geometric perception is an important prerequisite for a wide range of spatial AI systems. While state-of-the-art methods depend on large-scale training data, acquiring consistent and precise 3D annotations from in-the-wild videos remains a key challenge. In this work, we introduce ViPE, a handy and versatile video processing engine designed to bridge this gap. ViPE efficiently estimates camera intrinsics, camera motion, and dense, near-metric depth maps from unconstrained raw videos. It is robust to diverse scenarios, including dynamic selfie videos, cinematic shots, or dashcams, and supports various camera models such as pinhole, wide-angle, and 360deg panoramas. We have benchmarked ViPE on multiple benchmarks. Notably, it outperforms existing uncalibrated pose estimation baselines by 18%/50% on TUM/KITTI sequences, and runs at 3-5FPS on a single GPU for standard input resolutions. We use ViPE to annotate a large-scale collection of videos. This collection includes around 100K real-world internet videos, 1M high-quality AI-generated videos, and 2K panoramic videos, totaling approximately 96M frames -- all annotated with accurate camera poses and dense depth maps. We open-source ViPE and the annotated dataset with the hope of accelerating the development of spatial AI systems.

Key facts. Cited by 170 papers in the map (data through 14 Oct 2026). Builds on 78 of its 92 references that are in the map. First posted on arXiv 12 Aug 2025.

Builds on

78 of 92 references.

References · 78

  1. 2023Segment Anythingcited by 8,234
  2. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  3. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
  4. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,567
  5. 2025VGGT: Visual Geometry Grounded Transformercited by 1,539
  6. 2024Depth Anything V2cited by 1,422
  7. 2015ORB-SLAM: a Versatile and Accurate Monocular SLAM Systemcited by 988
  8. 2024Grounding Image Matching in 3D with MASt3Rcited by 983
  9. 2020ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAMcited by 878
  10. 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
  11. 2021DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Camerascited by 687
  12. 2023LightGlue: Local Feature Matching at Light Speedcited by 590
38 more
  1. 2025Continuous 3D Perception Model with Persistent Statecited by 548
  2. 2016Direct Sparse Odometrycited by 485
  3. 2021ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Datacited by 454
  4. 2024MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motioncited by 442
  5. 2022XMem: Long-Term Video Object Segmentation with an Atkinson-Shiffrin Memory Modelcited by 439
  6. 2024CAT3D: Create Anything in 3D with Multi-View Diffusion Modelscited by 432
  7. 2024Depth Pro: Sharp Monocular Metric Depth in Less Than a Secondcited by 404
  8. 2024Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimationcited by 395
  9. 2024Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teacherscited by 393
  10. 2022Kubric: A scalable dataset generatorcited by 381
  11. 2025Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Passcited by 367
  12. 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 341
  13. 2025π3: Permutation-Equivariant Visual Geometry Learningcited by 319
  14. 20243D Reconstruction with Spatial Memorycited by 277
  15. 2024MegaSaM: Accurate, Fast, and Robust Structure and Motion from Casual Dynamic Videoscited by 228
  16. 2023Visual Geometry Grounded Deep Structure From Motioncited by 227
  17. 2024Global Structure-from-Motion Revisitedcited by 218
  18. 2025Video Depth Anything: Consistent Depth Estimation for Super-Long Videoscited by 207
  19. 2024MASt3R-SLAM: Real-Time Dense SLAM with 3D Reconstruction Priorscited by 205
  20. 2023Segment and Track Anythingcited by 193
  21. 2024LVSM: A Large View Synthesis Model with Minimal 3D Inductive Biascited by 189
  22. 2025Stable Virtual Camera: Generative View Synthesis with Diffusion Modelscited by 182
  23. 2025UniDepthV2: Universal Monocular Metric Depth Estimation Made Simplercited by 173
  24. 2024MASt3R-SfM: a Fully-Integrated Solution for Unconstrained Structure-from-Motioncited by 162
  25. 2024CAT4D: Create Anything in 4D with Multi-View Video Diffusion Modelscited by 161
  26. 2024GenAD: Generalized Predictive Model for Autonomous Drivingcited by 150
  27. 2024MV-DUSt3R+: Single-Stage Scene Reconstruction from Sparse Views In 2 Secondscited by 141
  28. 2025MUSt3R: Multi-view Network for Stereo 3D Reconstructioncited by 141
  29. 2025VGGT-SLAM: Dense RGB SLAM Optimized on the SL(4) Manifoldcited by 136
  30. 2025TrajectoryCrafter: Redirecting Camera Trajectory for Monocular Videos via Diffusion Modelscited by 125
  31. 2025Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal Controlcited by 106
  32. 2023Neural Kernel Surface Reconstructioncited by 104
  33. 2024SLAM3R: Real-Time Dense Scene Reconstruction from Monocular RGB Videoscited by 102
  34. 2025Aether: Geometric-Aware Unified World Modelingcited by 97
  35. 2022ParticleSfM: Exploiting Dense Point Trajectories for Localizing Moving Cameras in the Wildcited by 92
  36. 2025Easi3R: Estimating Disentangled Motion from DUSt3R Without Trainingcited by 91
  37. 2024Stereo4D: Learning How Things Move in 3D from Internet Stereo Videoscited by 89
  38. 2025St4RTrack: Simultaneous 4D Reconstruction and Tracking in the Worldcited by 89

50 most cited of 78 All 78 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 38; 2026: 132.

170 papers.

Sort:

By area:

  1. 2025World Simulation with Video Foundation Models for Physical AIcited by 200
  2. 2025WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modelingcited by 170
  3. 2026Cosmos 3: Omnimodal World Models for Physical AIcited by 165
  4. 2025TTT3R: 3D Reconstruction as Test-Time Trainingcited by 129
  5. 2026Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memorycited by 100
  6. 2025RELIC: Interactive Video World Model with Long-Horizon Memorycited by 96
  7. 2025SpatialVID: A Large-Scale Video Dataset with Spatial Annotationscited by 77
  8. 2025Hunyuan-GameCraft-2: Instruction-following Interactive Game World Modelcited by 61
  9. 2026SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformercited by 55
  10. 2026Lyra 2.0: Explorable Generative 3D Worldscited by 44
  11. 2025Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillationcited by 41
  12. 2025Unified Camera Positional Encoding for Controlled Video Generationcited by 37
38 more
  1. 2026LingBot-Map: Geometric Context Transformer for Streaming 3D Reconstructioncited by 37
  2. 2026MIND: Benchmarking Memory Consistency and Action Control in World Modelscited by 35
  3. 20254DNeX: Feed-Forward 4D Generative Modeling Made Easycited by 34
  4. 2025FlashWorld: High-quality 3D Scene Generation within Secondscited by 32
  5. 2025ChronoEdit: Towards Temporal Reasoning for Image Editing and World Simulationcited by 29
  6. 2025What Is The Best 3D Scene Representation for Robotics? From Geometric to Foundation Modelscited by 29
  7. 2026MosaicMem: Hybrid Spatial Memory for Controllable Video World Modelscited by 27
  8. 2026WorldMark: A Unified Benchmark Suite for Interactive Video World Modelscited by 26
  9. 2026VGGT-Omegacited by 26
  10. 2025Generative Video Motion Editing with 3D Point Trackscited by 23
  11. 2025AMB3R: Accurate Feed-forward Metric-scale 3D Reconstruction with Backendcited by 20
  12. 2026WorldStereo: Bridging Camera-Guided Video Generation and Scene Reconstruction via 3D Geometric Memoriescited by 18
  13. 2026Latent Spatial Memory for Video World Modelscited by 17
  14. 2025Latent Chain-of-Thought World Modeling for End-to-End Drivingcited by 15
  15. 2026Wonder: Video World Model Done Bettercited by 15
  16. 2025Do You Know Where Your Camera Is? View-Invariant Policy Learning with Camera Conditioningcited by 14
  17. 2025BulletTime: Decoupled Control of Time and Camera Pose for Video Generationcited by 14
  18. 2026Physical Simulator In-the-Loop Video Generationcited by 14
  19. 2026EgoSim: Egocentric World Simulator for Embodied Interaction Generationcited by 14
  20. 2026iWorld-Bench: A Benchmark for Interactive World Models with a Unified Action Generation Frameworkcited by 14
  21. 2026ActWorld: From Explorable to Interactive World Model via Action-Aware Memorycited by 14
  22. 2025Reinforcement Learning with Inverse Rewards for World Model Post-trainingcited by 13
  23. 2026ReRoPE: Repurposing RoPE for Relative Camera Controlcited by 13
  24. 2026Track4World: Feedforward World-centric Dense 3D Tracking of All Pixelscited by 13
  25. 2026WildWorld: A Large-Scale Dataset for Dynamic World Modeling with Actions and Explicit State toward Generative ARPGcited by 13
  26. 2026Geometry-Aware Implicit Memory for Video World Modelscited by 13
  27. 2026EchoWM: Open and Enterable Omnimodal World Modelscited by 13
  28. 2026AnyView: Synthesizing Any Novel View in Dynamic Scenescited by 12
  29. 2026Thinking in Dynamics: How Multimodal Large Language Models Perceive, Track, and Reason Dynamics in Physical 4D Worldcited by 12
  30. 2026Kinema4D: Kinematic 4D World Modeling for Spatiotemporal Embodied Simulationcited by 12
  31. 2026WorldCam: Interactive Autoregressive 3D Gaming Worlds with Camera Pose as a Unifying Geometric Representationcited by 12
  32. 2025ReDirector: Creating Any-Length Video Retakes with Rotary Camera Encodingcited by 11
  33. 2026AlayaWorld: Interactive Long-Horizon World Modeling -- Full Technical Reportcited by 11
  34. 2025Taming Camera-Controlled Video Generation with Verifiable Geometry Rewardcited by 10
  35. 2026Olaf-World: Orienting Latent Actions for Video World Modelingcited by 10
  36. 2026ACE-Ego-0: Unifying Egocentric Human and Robotic Data for VLA Pretrainingcited by 10
  37. 2026WorldRoamBench: An Open-World Benchmark for Long-Horizon Stability of Interactive World Modelscited by 10
  38. 2026ReWorld: An Interactive World Model with Long-Horizon Memorycited by 10

50 most cited of 170 All 170 (JSON)