mlsanityData to 14 Oct 2026

GRIN: Zero-Shot Metric Depth with Pixel-Level Diffusion

arXiv 2409.09896 ↗First posted 15 Sep 2024cs.CV cs.LG eess.IVcited by 6

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

3D reconstruction from a single image is a long-standing problem in computer vision. Learning-based methods address its inherent scale ambiguity by leveraging increasingly large labeled and unlabeled datasets, to produce geometric priors capable of generating accurate predictions across domains. As a result, state of the art approaches show impressive performance in zero-shot relative and metric depth estimation. Recently, diffusion models have exhibited remarkable scalability and generalizable properties in their learned representations. However, because these models repurpose tools originally designed for image generation, they can only operate on dense ground-truth, which is not available for most depth labels, especially in real-world settings. In this paper we present GRIN, an efficient diffusion model designed to ingest sparse unstructured training data. We use image features with 3D geometric positional encodings to condition the diffusion process both globally and locally, generating depth predictions at a pixel-level. With comprehensive experiments across eight indoor and outdoor datasets, we show that GRIN establishes a new state of the art in zero-shot metric monocular depth estimation even when trained from scratch.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 67 of its 85 references that are in the map. First posted on arXiv 15 Sep 2024.

Builds on

67 of 85 references.

Closest ancestors · 5

Same direction, newest first

  1. 2023Zero-Shot Metric Depth with a Field-of-View Conditioned Diffusion Model20 Dec
  2. 2023Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation4 Dec
  3. 2023The Surprising Effectiveness of Diffusion Models for Optical Flow and Monocular Depth Estimation2 Jun
  4. 2023Zero-1-to-3: Zero-shot One Image to 3D Object20 Mar
  5. 2023Monocular Depth Estimation using Diffusion Models28 Feb

Other references · 62

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  6. 2020Denoising Diffusion Implicit Modelscited by 9,781
  7. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  8. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  9. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  10. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  11. 2017Neural Discrete Representation Learningcited by 4,712
  12. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
38 more
  1. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
  2. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  3. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  4. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,495
  5. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,492
  6. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,360
  7. 2021Palette: Image-to-Image Diffusion Modelscited by 1,300
  8. 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,228
  9. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,163
  10. 2023One-step Diffusion with Distribution Matching Distillationcited by 1,019
  11. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 782
  12. 2018Deep Ordinal Regression Network for Monocular Depth Estimationcited by 723
  13. 2023Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecastingcited by 707
  14. 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 657
  15. 2020AdaBins: Depth Estimation using Adaptive Binscited by 601
  16. 2020TartanAir: A Dataset to Push the Limits of Visual SLAMcited by 506
  17. 2014Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecturecited by 483
  18. 20193D Packing for Self-Supervised Monocular Depth Estimationcited by 439
  19. 2019From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimationcited by 382
  20. 2024UniDepth: Universal Monocular Metric Depth Estimationcited by 382
  21. 2023Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Imagecited by 365
  22. 2021Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scanscited by 336
  23. 2023Unleashing Text-to-Image Diffusion Models for Visual Perceptioncited by 269
  24. 2019DIODE: A Dense Indoor and Outdoor DEpth Datasetcited by 240
  25. 2020One Thousand and One Hours: Self-driving Motion Prediction Datasetcited by 240
  26. 2022Scalable Adaptive Computation for Iterative Generationcited by 195
  27. 2021The Temporal Opportunist: Self-Supervised Multi-Frame Monocular Depthcited by 190
  28. 2019Depth from Videos in the Wild: Unsupervised Monocular Depth Learning from Unknown Camerascited by 170
  29. 2022NeW CRFs: Neural Window Fully-connected CRFs for Monocular Depth Estimationcited by 153
  30. 2023DDP: Diffusion Model for Dense Visual Predictioncited by 148
  31. 2020Feature-metric Loss for Self-supervised Learning of Depth and Egomotioncited by 140
  32. 2023Towards Zero-Shot Scale-Aware Monocular Depth Estimationcited by 101
  33. 2022A Generalist Framework for Panoptic Segmentation of Images and Videoscited by 99
  34. 2023DiffusionDepth: Diffusion Denoising Approach for Monocular Depth Estimationcited by 82
  35. 2022SurroundDepth: Entangling Surrounding Views for Self-Supervised Multi-Camera Depth Estimationcited by 76
  36. 2022Multi-Frame Self-Supervised Depth with Transformerscited by 64
  37. 2023On the Design Fundamentals of Diffusion Models: A Surveycited by 60
  38. 2021Towards Real-Time Monocular Depth Estimation for Robotics: A Surveycited by 57

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 2.

6 papers.

  1. 2025MVSAnywhere: Zero-Shot Multi-View Stereocited by 26
  2. 2025Zero-Shot Novel View and Depth Synthesis with Multi-View Geometric Diffusioncited by 12
  3. 2025Hybrid-grained Feature Aggregation with Coarse-to-fine Language Guidance for Self-supervised Monocular Depth Estimationcited by 6
  4. 2026Spatially Aware World Action Model via Geometric Latent Diffusioncited by 5
  5. 2025PFDepth: Heterogeneous Pinhole-Fisheye Joint Depth Estimation via Distortion-aware Gaussian-Splatted Volumetric Fusioncited by 1
  6. 2026Zero-Shot Novel Depth Synthesis Using 3D Foundation Models Scene Representationscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025GeoDiff: Geometry-Guided Diffusion for Metric Depth Estimation
  2. 2024MetricGold: Leveraging Text-To-Image Latent Diffusion Models for Metric Depth Estimation
  3. 2024Marigold-DC: Zero-Shot Monocular Depth Completion with Guided Diffusion
  4. 2024Diffusion Models for Monocular Depth Estimation: Overcoming Challenging Conditions
  5. 2026Need for Speed: Zero-Shot Depth Completion with Single-Step Diffusion
  6. 2025Zero-shot Depth Completion via Test-time Alignment with Affine-invariant Depth Prior
  7. 2026Marigold V2: Revisiting Diffusion Transformers for Monocular Depth Estimation
  8. 2024SharpDepth: Sharpening Metric Depth Predictions Using Diffusion Distillation