mlsanityData to 7 Oct 2026

GeometryCrafter: Consistent Geometry Estimation for Open-world Videos with Diffusion Priors

arXiv 2504.01016 ↗First posted 1 Apr 2025cs.GR cs.AI cs.CVcited by 36

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

Despite remarkable advancements in video depth estimation, existing methods exhibit inherent limitations in achieving geometric fidelity through the affine-invariant predictions, limiting their applicability in reconstruction and other metrically grounded downstream tasks. We propose GeometryCrafter, a novel framework that recovers high-fidelity point map sequences with temporal coherence from open-world videos, enabling accurate 3D/4D reconstruction, camera parameter estimation, and other depth-based applications. At the core of our approach lies a point map Variational Autoencoder (VAE) that learns a latent space agnostic to video latent distributions for effective point map encoding and decoding. Leveraging the VAE, we train a video diffusion model to model the distribution of point map sequences conditioned on the input videos. Extensive evaluations on diverse datasets demonstrate that GeometryCrafter achieves state-of-the-art 3D accuracy, temporal consistency, and generalization capability.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 79 of its 106 references that are in the map. First posted on arXiv 1 Apr 2025.

Builds on

79 of 106 references.

Closest ancestors · 9

Same direction, newest first

  1. 2024SharpDepth: Sharpening Metric Depth Predictions Using Diffusion Distillation27 Nov
  2. 2024DimensionX: Create Any 3D and 4D Scenes from a Single Image with Controllable Video Diffusion7 Nov
  3. 2024Depth Any Video with Scalable Synthetic Data14 Oct
  4. 2024Fine-Tuning Image-Conditional Diffusion Models is Easier than You Think17 Sep
  5. 2024StereoCrafter: Diffusion-based Generation of Long and High-fidelity Stereoscopic 3D from Monocular Videos11 Sep
  6. 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis3 Sep
  7. 2024Learning Temporally Consistent Video Depth from Video Diffusion Priors3 Jun
  8. 2024DepthFM: Fast Monocular Depth Estimation with Flow Matching20 Mar
  9. 2024GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Image18 Mar

Other references · 70

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2023Segment Anythingcited by 8,158
  4. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  6. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  7. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  8. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  9. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  10. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
  11. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
  12. 2024Depth Anything V2cited by 1,385
38 more
  1. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,346
  2. 2017SuperPoint: Self-Supervised Interest Point Detection and Descriptioncited by 1,313
  3. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,264
  4. 2015A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimationcited by 878
  5. 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 747
  6. 2018Deep Ordinal Regression Network for Monocular Depth Estimationcited by 721
  7. 2020AdaBins: Depth Estimation using Adaptive Binscited by 596
  8. 2020Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understandingcited by 593
  9. 2023DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Visioncited by 514
  10. 2020TartanAir: A Dataset to Push the Limits of Visual SLAMcited by 494
  11. 2018Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Drivingcited by 455
  12. 20193D Packing for Self-Supervised Monocular Depth Estimationcited by 437
  13. 2022XMem: Long-Term Video Object Segmentation with an Atkinson-Shiffrin Memory Modelcited by 436
  14. 2024MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motioncited by 431
  15. 2024Depth Pro: Sharp Monocular Metric Depth in Less Than a Secondcited by 395
  16. 2024Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimationcited by 389
  17. 2018HiDDeN: Hiding Data With Deep Networkscited by 384
  18. 2019From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimationcited by 379
  19. 2024UniDepth: Universal Monocular Metric Depth Estimationcited by 375
  20. 2023Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Imagecited by 357
  21. 2020Virtual KITTI 2cited by 345
  22. 2024MoGe: Unlocking Accurate Monocular Geometry Estimation for Open-Domain Images with Optimal Training Supervisioncited by 290
  23. 2019Structured3D: A Large Photo-realistic Dataset for Structured 3D Modelingcited by 285
  24. 2018DeepMVS: Learning Multi-view Stereopsiscited by 277
  25. 2021Is Pseudo-Lidar needed for Monocular 3D Object detection?cited by 248
  26. 2024Shape of Motion: 4D Reconstruction from a Single Videocited by 247
  27. 2019DIODE: A Dense Indoor and Outdoor DEpth Datasetcited by 236
  28. 2020Consistent Video Depth Estimationcited by 220
  29. 2020Learning to Recover 3D Scene Shape from a Single Imagecited by 214
  30. 2024DepthCrafter: Generating Consistent Long Depth Sequences for Open-world Videoscited by 202
  31. 2020Robust Consistent Video Depth Estimationcited by 188
  32. 2023MatrixCity: A Large-scale City Dataset for City-scale Neural Rendering and Beyondcited by 185
  33. 2024SpatialTracker: Tracking Any 2D Pixels in 3D Spacecited by 178
  34. 2022Depth-Aware Generative Adversarial Network for Talking Head Video Generationcited by 177
  35. 2024MoSca: Dynamic Gaussian Fusion from Casual Videos via 4D Motion Scaffoldscited by 177
  36. 2023Spring: A High-Resolution High-Detail Dataset and Benchmark for Scene Flow, Optical Flow and Stereocited by 170
  37. 2022BinsFormer: Revisiting Adaptive Bins for Monocular Depth Estimationcited by 166
  38. 2024Lotus: Diffusion-based Visual Foundation Model for High-quality Dense Predictioncited by 163

50 most cited of 70 All 70 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 16; 2026: 20.

36 papers.

Sort:

By area:

  1. 2025ViPE: Video Pose Engine for 3D Geometric Perceptioncited by 162
  2. 2025Geo4D: Leveraging Video Generators for Geometric 4D Scene Reconstructioncited by 66
  3. 2025Pixel-Perfect Depth with Semantics-Prompted Diffusion Transformerscited by 47
  4. 2025MoVieS: Motion-Aware 4D Dynamic View Synthesis in One Secondcited by 31
  5. 20254DNeX: Feed-Forward 4D Generative Modeling Made Easycited by 31
  6. 2026Vista4D: Video Reshooting with 4D Point Cloudscited by 15
  7. 2025Reconstructing 4D Spatial Intelligence: A Surveycited by 13
  8. 2026Mesh4D: 4D Mesh Reconstruction and Tracking from Monocular Videocited by 12
  9. 2026Track4World: Feedforward World-centric Dense 3D Tracking of All Pixelscited by 10
  10. 2025VGGT-X: When VGGT Meets Dense Novel View Synthesiscited by 8
  11. 2025PAGE-4D: Disentangled pose and geometry estimation for vggt-4d perceptioncited by 7
  12. 2026Geometry-as-context: Modulating Explicit 3D in Scene-consistent Video Generation to Geometry Contextcited by 7
24 more
  1. 2025E3D-Bench: A Benchmark for End-to-End 3D Geometric Foundation Modelscited by 6
  2. 2025CtrlVDiff: Controllable Video Generation via Unified Multimodal Video Diffusioncited by 5
  3. 2026PointDiT: Pixel-Space Diffusion for Monocular Geometry Estimationcited by 5
  4. 2025ColonCrafter: A Depth Estimation Model for Colonoscopy Videos Using Diffusion Priorscited by 3
  5. 2025WorldReel: 4D Video Generation with Consistent Geometry and Motion Modelingcited by 3
  6. 2026MotionCrafter: Dense Geometry and Motion Reconstruction with a 4D VAEcited by 3
  7. 2026DVD: Deterministic Video Depth Estimation with Generative Priorscited by 3
  8. 2026SLARM: Streaming and Language-Aligned Reconstruction Model for Dynamic Scenescited by 3
  9. 2026UniVidX: A Unified Multimodal Framework for Versatile Video Generation via Diffusion Priorscited by 3
  10. 2026TrackCraft3R: Repurposing Video Diffusion Transformers for Dense 3D Trackingcited by 3
  11. 2026GenRecon: Bridging Generative Priors for Multi-View 3D Scene Reconstructioncited by 3
  12. 2026Towards Consistent Video Geometry Estimationcited by 3
  13. 2026HarmoHOI: Harmonizing Appearance and 3D Motion for Multi-view Hand-Object Interaction Synthesiscited by 3
  14. 2025CloseUpShot: Close-up Novel View Synthesis from Sparse-views via Point-conditioned Diffusion Modelcited by 2
  15. 2025HuPrior3R: Incorporating Human Priors for Better 3D Dynamic Reconstruction from Monocular Videoscited by 2
  16. 2026DAGE: Dual-Stream Architecture for Efficient and Fine-Grained Geometry Estimationcited by 2
  17. 2026Reshoot-Anything: A Self-Supervised Model for In-the-Wild Video Reshootingcited by 2
  18. 2026Lift4D: Harmonizing Single-View 3D Estimation for 4D Reconstruction In-the-Wildcited by 2
  19. 2026Pano2World: End-to-End 3D Generation via Unified Multi-View Sequencescited by 2
  20. 2025SyncMV4D: Synchronized Multi-view Joint Diffusion of Appearance and Motion for Hand-Object Interaction Synthesiscited by 1
  21. 2025WonderZoom: Multi-Scale 3D World Generationcited by 1
  22. 2026Fast 4D Mesh Generation by Spatio-Temporal Attention Chainscited by 1
  23. 2026Beyond Inpainting: Unleash 3D Understanding for Precise Camera-Controlled Video Generationcited by 0
  24. 2026Beyond Pixels: From Video Priors to 4D Worldscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025SpatialCrafter: Unleashing the Imagination of Video Diffusion Models for Scene Reconstruction from Limited Observations
  2. 2026VideoWeave: Unlocking Geometric Consistency in Video Generation via Joint Geometry-Video Modeling
  3. 2025Can Video Diffusion Model Reconstruct 4D Geometry?
  4. 2025GeoVideo: Introducing Geometric Regularization into Video Generation Model
  5. 2026Beyond Gaussian Bottlenecks: Topologically Aligned Encoding of Vision-Transformer Feature Spaces
  6. 2025Geometry Forcing: Marrying Video Diffusion and 3D Representation for Consistent World Modeling
  7. 2026Video Generative Models as Geometry Learner
  8. 2025UniGeo: Taming Video Diffusion for Unified Consistent Geometry Estimation