mlsanityData to 7 Oct 2026

ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depth

arXiv 2302.12288 ↗First posted 23 Feb 2023cs.CVcited by 646

Not on the map. Why?

Cited by 77 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

This paper tackles the problem of depth estimation from a single image. Existing work either focuses on generalization performance disregarding metric scale, i.e. relative depth estimation, or state-of-the-art results on specific datasets, i.e. metric depth estimation. We propose the first approach that combines both worlds, leading to a model with excellent generalization performance while maintaining metric scale. Our flagship model, ZoeD-M12-NK, is pre-trained on 12 datasets using relative depth and fine-tuned on two datasets using metric depth. We use a lightweight head with a novel bin adjustment design called metric bins module for each domain. During inference, each input image is automatically routed to the appropriate head using a latent classifier. Our framework admits multiple configurations depending on the datasets used for relative depth pre-training and metric fine-tuning. Without pre-training, we can already significantly improve the state of the art (SOTA) on the NYU Depth v2 indoor dataset. Pre-training on twelve datasets and fine-tuning on the NYU Depth v2 indoor dataset, we can further improve SOTA for a total of 21% in terms of relative absolute error (REL). Finally, ZoeD-M12-NK is the first model that can jointly train on multiple datasets (NYU Depth v2 and KITTI) without a significant drop in performance and achieve unprecedented zero-shot generalization performance to eight unseen datasets from both indoor and outdoor domains. The code and pre-trained models are publicly available at https://github.com/isl-org/ZoeDepth .

Key facts. Cited by 646 papers in the map (data through 7 Oct 2026). Builds on 37 of its 51 references that are in the map. First posted on arXiv 23 Feb 2023.

Builds on

37 of 51 references.

References · 37

  1. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  2. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  3. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  4. 2021Vision Transformers for Dense Predictioncited by 1,789
  5. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
  6. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,264
  7. 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,215
  8. 2018MegaDepth: Learning Single-View Depth Prediction from Internet Photoscited by 731
  9. 2018Deep Ordinal Regression Network for Monocular Depth Estimationcited by 721
  10. 2020AdaBins: Depth Estimation using Adaptive Binscited by 596
  11. 2020Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understandingcited by 593
  12. 2020TartanAir: A Dataset to Push the Limits of Visual SLAMcited by 494
25 more
  1. 2019BlendedMVS: A Large-scale Dataset for Generalized Multi-view Stereo Networkscited by 487
  2. 20193D Packing for Self-Supervised Monocular Depth Estimationcited by 437
  3. 2016Deeper Depth Prediction with Fully Convolutional Residual Networkscited by 402
  4. 2019From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimationcited by 379
  5. 2020Virtual KITTI 2cited by 345
  6. 2019DIODE: A Dense Indoor and Outdoor DEpth Datasetcited by 236
  7. 2019Enforcing geometric constraints of virtual normal for depth predictioncited by 233
  8. 2020Learning to Recover 3D Scene Shape from a Single Imagecited by 214
  9. 2018The ApolloScape Open Dataset for Autonomous Driving and its Applicationcited by 188
  10. 2022BinsFormer: Revisiting Adaptive Bins for Monocular Depth Estimationcited by 166
  11. 2022A Review of Sparse Expert Models in Deep Learningcited by 147
  12. 2018Revisiting Single Image Depth Estimation: Toward Higher Resolution Maps with Accurate Object Boundariescited by 144
  13. 2018Evaluation of CNN-based Single-Image Depth Estimation Methodscited by 137
  14. 2022LocalBins: Improving Depth Estimation by Learning Local Distributionscited by 92
  15. 2022Attention Attention Everywhere: Monocular Depth Prediction with Skip Attentioncited by 92
  16. 2019Web Stereo Video Supervision for Depth Prediction from Dynamic Scenescited by 73
  17. 2019SharpNet: Fast and Accurate Recovery of Occluding Contours in Monocular Depth Estimationcited by 59
  18. 2019Structure-Aware Residual Pyramid Network for Monocular Depth Estimationcited by 49
  19. 2018Deep attention-based classification network for robust depth predictioncited by 36
  20. 2021Single Image Depth Estimation: An Overviewcited by 35
  21. 2018Detail Preserving Depth Estimation from a Single Image Using Attention Guided Networkscited by 31
  22. 2017Unimodal probability distributions for deep ordinal classificationcited by 28
  23. 2017Monocular Depth Estimation with Hierarchical Fusion of Dilated CNNs and Soft-Weighted-Sum Inferencecited by 24
  24. 2022Depth Map Decomposition for Monocular Depth Estimationcited by 16
  25. 2019Deep Robust Single Image Depth Estimation Neural Network Using Scene Understandingcited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 71; 2024: 221; 2025: 230; 2026: 124.

646 papers.

Sort:

By area:

  1. 2024Depth Anything V2cited by 1,385
  2. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,346
  3. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 679
  4. 2025Continuous 3D Perception Model with Persistent Statecited by 527
  5. 2025SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Modelcited by 502
  6. 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 417
  7. 2024Depth Pro: Sharp Monocular Metric Depth in Less Than a Secondcited by 395
  8. 2024Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimationcited by 389
  9. 2024UniDepth: Universal Monocular Metric Depth Estimationcited by 375
  10. 20243D-VLA: A 3D Vision-Language-Action Generative World Modelcited by 320
  11. 2024SpatialRGPT: Grounded Spatial Reasoning in Vision Language Modelscited by 297
  12. 2024MoGe: Unlocking Accurate Monocular Geometry Estimation for Open-Domain Images with Optimal Training Supervisioncited by 290
38 more
  1. 2024DepthSplat: Connecting Gaussian Splatting and Depthcited by 275
  2. 2023Unleashing Text-to-Image Diffusion Models for Visual Perceptioncited by 266
  3. 2025MoGe-2: Accurate Monocular Geometry with Metric Scale and Sharp Detailscited by 250
  4. 2024Sapiens: Foundation for Human Vision Modelscited by 229
  5. 2023LucidDreamer: Domain-free Generation of 3D Gaussian Splatting Scenescited by 228
  6. 2024DepthCrafter: Generating Consistent Long Depth Sequences for Open-world Videoscited by 202
  7. 2025Video Depth Anything: Consistent Depth Estimation for Super-Long Videoscited by 202
  8. 2023VLFM: Vision-Language Frontier Maps for Zero-Shot Semantic Navigationcited by 201
  9. 2023COLMAP-Free 3D Gaussian Splattingcited by 193
  10. 2024SpatialTracker: Tracking Any 2D Pixels in 3D Spacecited by 178
  11. 2024MoSca: Dynamic Gaussian Fusion from Casual Videos via 4D Motion Scaffoldscited by 177
  12. 2024Probing the 3D Awareness of Visual Foundation Modelscited by 175
  13. 2023MiDaS v3.1 -- A Model Zoo for Robust Monocular Relative Depth Estimationcited by 173
  14. 2023Depth-Regularized Optimization for 3D Gaussian Splatting in Few-Shot Imagescited by 172
  15. 2025UniDepthV2: Universal Monocular Metric Depth Estimation Made Simplercited by 170
  16. 2025FoundationStereo: Zero-Shot Stereo Matchingcited by 168
  17. 2024SpatialBot: Precise Spatial Understanding with Vision Language Modelscited by 146
  18. 2025WorldScore: A Unified Evaluation Benchmark for World Generationcited by 143
  19. 2024DN-Splatter: Depth and Normal Priors for Gaussian Splatting and Meshingcited by 136
  20. 2024Training-free Camera Control for Video Generationcited by 119
  21. 2024RealmDreamer: Text-Driven 3D Scene Generation with Inpainting and Depth Diffusioncited by 109
  22. 2024TRAM: Global Trajectory and Motion of 3D Humans from in-the-wild Videoscited by 105
  23. 2025CAST: Component-Aligned 3D Scene Reconstruction from an RGB Imagecited by 101
  24. 2023Towards Zero-Shot Scale-Aware Monocular Depth Estimationcited by 100
  25. 2024Learning Temporally Consistent Video Depth from Video Diffusion Priorscited by 100
  26. 2024DiLightNet: Fine-grained Lighting Control for Diffusion-based Image Generationcited by 93
  27. 2023Language Embedded Radiance Fields for Zero-Shot Task-Oriented Graspingcited by 91
  28. 2025TesserAct: Learning 4D Embodied World Modelscited by 90
  29. 2025St4RTrack: Simultaneous 4D Reconstruction and Tracking in the Worldcited by 88
  30. 2025SpatialTrackerV2: 3D Point Tracking Made Easycited by 84
  31. 2024Fine-Tuning Image-Conditional Diffusion Models is Easier than You Thinkcited by 82
  32. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applicationscited by 82
  33. 2024General Flow as Foundation Affordance for Scalable Robot Learningcited by 80
  34. 2024Align3R: Aligned Monocular Depth Estimation for Dynamic Videoscited by 79
  35. 2025Marigold: Affordable Adaptation of Diffusion-Based Image Generators for Image Analysiscited by 78
  36. 2024DepthFM: Fast Monocular Depth Estimation with Flow Matchingcited by 76
  37. 2024Scene Coordinate Reconstruction: Posing of Image Collections via Incremental Learning of a Relocalizercited by 73
  38. 2024Prompting Depth Anything for 4K Resolution Accurate Metric Depth Estimationcited by 73

50 most cited of 646 All 646 (JSON)