mlsanityData to 7 Oct 2026

Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimation

arXiv 2404.15506 ↗First posted 22 Mar 2024cs.CVcited by 389

Not on the map. Why?

Cited by 40 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce Metric3D v2, a geometric foundation model for zero-shot metric depth and surface normal estimation from a single image, which is crucial for metric 3D recovery. While depth and normal are geometrically related and highly complimentary, they present distinct challenges. SoTA monocular depth methods achieve zero-shot generalization by learning affine-invariant depths, which cannot recover real-world metrics. Meanwhile, SoTA normal estimation methods have limited zero-shot performance due to the lack of large-scale labeled data. To tackle these issues, we propose solutions for both metric depth estimation and surface normal estimation. For metric depth estimation, we show that the key to a zero-shot single-view model lies in resolving the metric ambiguity from various camera models and large-scale data training. We propose a canonical camera space transformation module, which explicitly addresses the ambiguity problem and can be effortlessly plugged into existing monocular models. For surface normal estimation, we propose a joint depth-normal optimization module to distill diverse data knowledge from metric depth, enabling normal estimators to learn beyond normal labels. Equipped with these modules, our depth-normal models can be stably trained with over 16 million of images from thousands of camera models with different-type annotations, resulting in zero-shot generalization to in-the-wild images with unseen camera settings. Our method enables the accurate recovery of metric 3D structures on randomly collected internet images, paving the way for plausible single-image metrology. Our project page is at https://JUGGHM.github.io/Metric3Dv2.

Key facts. Cited by 389 papers in the map (data through 7 Oct 2026). Builds on 101 of its 137 references that are in the map. First posted on arXiv 22 Mar 2024.

Builds on

101 of 137 references.

References · 101

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  5. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  6. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  7. 2022A ConvNet for the 2020scited by 3,545
  8. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  9. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
  10. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  11. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  12. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
38 more
  1. 2021Vision Transformers for Dense Predictioncited by 1,789
  2. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
  3. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,346
  4. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,264
  5. 2021Scaling Vision Transformerscited by 993
  6. 2017PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volumecited by 966
  7. 2019The Replica Dataset: A Digital Replica of Indoor Spacescited by 885
  8. 2023Vision Transformers Need Registerscited by 844
  9. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  10. 2016ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Camerascited by 801
  11. 2021Depth-supervised NeRF: Fewer Views and Faster Training for Freecited by 732
  12. 2023Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecastingcited by 701
  13. 2021DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Camerascited by 673
  14. 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 649
  15. 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 646
  16. 2018Taskonomy: Disentangling Task Transfer Learningcited by 631
  17. 2020AdaBins: Depth Estimation using Adaptive Binscited by 596
  18. 2020Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understandingcited by 593
  19. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
  20. 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 568
  21. 2020PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitizationcited by 521
  22. 2014Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecturecited by 483
  23. 2022MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstructioncited by 467
  24. 20193D Packing for Self-Supervised Monocular Depth Estimationcited by 437
  25. 2016Deeper Depth Prediction with Fully Convolutional Residual Networkscited by 402
  26. 2019From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimationcited by 379
  27. 2021RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matchingcited by 348
  28. 2020Virtual KITTI 2cited by 345
  29. 2018GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Posecited by 341
  30. 2021Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scanscited by 336
  31. 2023Repurposing Diffusion-Based Image Generators for Monocular Depth Estimationcited by 323
  32. 2021Dense Depth Priors for Neural Radiance Fields from Sparse Input Viewscited by 301
  33. 2021DSEC: A Stereo Event Camera Dataset for Driving Scenarioscited by 297
  34. 2022Unifying Flow, Stereo and Depth Estimationcited by 270
  35. 2019DIODE: A Dense Indoor and Outdoor DEpth Datasetcited by 236
  36. 2019Enforcing geometric constraints of virtual normal for depth predictioncited by 233
  37. 2023Uni3D: Exploring Unified 3D Representation at Scalecited by 231
  38. 2020Learning to Recover 3D Scene Shape from a Single Imagecited by 214

50 most cited of 101 All 101 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 58; 2025: 184; 2026: 147.

389 papers.

Sort:

By area:

  1. 2024Depth Anything V2cited by 1,385
  2. 2025Qwen-Image Technical Reportcited by 973
  3. 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 626
  4. 2025Continuous 3D Perception Model with Persistent Statecited by 527
  5. 2024Depth Pro: Sharp Monocular Metric Depth in Less Than a Secondcited by 395
  6. 2024SpatialRGPT: Grounded Spatial Reasoning in Vision Language Modelscited by 297
  7. 2025MapAnything: Universal Feed-Forward Metric 3D Reconstructioncited by 295
  8. 2024MoGe: Unlocking Accurate Monocular Geometry Estimation for Open-Domain Images with Optimal Training Supervisioncited by 290
  9. 2025MoGe-2: Accurate Monocular Geometry with Metric Scale and Sharp Detailscited by 250
  10. 2024DepthCrafter: Generating Consistent Long Depth Sequences for Open-world Videoscited by 202
  11. 2024MoSca: Dynamic Gaussian Fusion from Casual Videos via 4D Motion Scaffoldscited by 177
  12. 2025UniDepthV2: Universal Monocular Metric Depth Estimation Made Simplercited by 170
38 more
  1. 2024Lotus: Diffusion-based Visual Foundation Model for High-quality Dense Predictioncited by 163
  2. 2025ViPE: Video Pose Engine for 3D Geometric Perceptioncited by 162
  3. 2024DN-Splatter: Depth and Normal Priors for Gaussian Splatting and Meshingcited by 136
  4. 2024Flash3D: Feed-Forward Generalisable 3D Scene Reconstruction from a Single Imagecited by 113
  5. 2024TRAM: Global Trajectory and Motion of 3D Humans from in-the-wild Videoscited by 105
  6. 2024Learning Temporally Consistent Video Depth from Video Diffusion Priorscited by 100
  7. 2025Voyager: Long-Range and World-Consistent Video Diffusion for Explorable 3D Scene Generationcited by 93
  8. 2025World4Drive: End-to-End Autonomous Driving via Intention-aware Physical Latent World Modelcited by 91
  9. 2025SpatialTrackerV2: 3D Point Tracking Made Easycited by 84
  10. 2024Fine-Tuning Image-Conditional Diffusion Models is Easier than You Thinkcited by 82
  11. 2024Align3R: Aligned Monocular Depth Estimation for Dynamic Videoscited by 79
  12. 2025Marigold: Affordable Adaptation of Diffusion-Based Image Generators for Image Analysiscited by 78
  13. 2024Prompting Depth Anything for 4K Resolution Accurate Metric Depth Estimationcited by 73
  14. 2025UniK3D: Universal Camera Monocular 3D Estimationcited by 60
  15. 2024Gen3DSR: Generalizable 3D Scene Reconstruction via Divide and Conquer from a Single Viewcited by 59
  16. 2024Depth Any Video with Scalable Synthetic Datacited by 57
  17. 2024SCube: Instant Large-Scale Scene Reconstruction using VoxSplatscited by 56
  18. 2025RealCam-I2V: Real-World Image-to-Video Generation with Interactive Complex Camera Controlcited by 54
  19. 2025Multi-SpatialMLLM: Multi-Frame Spatial Understanding with Multi-Modal Large Language Modelscited by 52
  20. 2025WildGS-SLAM: Monocular Gaussian Splatting SLAM in Dynamic Environmentscited by 48
  21. 2025Pixel-Perfect Depth with Semantics-Prompted Diffusion Transformerscited by 47
  22. 2024GaussTR: Foundation Model-Aligned Gaussian Transformer for Self-Supervised 3D Spatial Understandingcited by 46
  23. 2025VidBot: Learning Generalizable 3D Actions from In-the-Wild 2D Human Videos for Zero-Shot Robotic Manipulationcited by 46
  24. 2024Stereo Anywhere: Robust Zero-Shot Deep Stereo Matching Even Where Either Stereo or Mono Failcited by 45
  25. 2025LONG3R: Long Sequence Streaming 3D Reconstructioncited by 45
  26. 2026Image Generators are Generalist Vision Learnerscited by 40
  27. 2025Matrix3D: Large Photogrammetry Model All-in-Onecited by 39
  28. 2024VistaDream: Sampling multiview consistent images for single-view scene reconstructioncited by 38
  29. 2025Depth Any Camera: Zero-Shot Metric Depth Estimation from Any Cameracited by 38
  30. 2025DICEPTION: A Generalist Diffusion Model for Visual Perceptual Taskscited by 37
  31. 2025LBM: Latent Bridge Matching for Fast Image-to-Image Translationcited by 36
  32. 2025GeometryCrafter: Consistent Geometry Estimation for Open-world Videos with Diffusion Priorscited by 36
  33. 2025From Seeing to Doing: Bridging Reasoning and Decision for Robotic Manipulationcited by 34
  34. 2024Omni-Scene: Omni-Gaussian Representation for Ego-Centric Sparse-View Scene Reconstructioncited by 32
  35. 2025Depth Anything with Any Priorcited by 32
  36. 2025MoVieS: Motion-Aware 4D Dynamic View Synthesis in One Secondcited by 31
  37. 2025DepthLM: Metric Depth From Vision Language Modelscited by 29
  38. 2024BetterDepth: Plug-and-Play Diffusion Refiner for Zero-Shot Monocular Depth Estimationcited by 28

50 most cited of 389 All 389 (JSON)