mlsanityData to 14 Oct 2026

MetricGold: Leveraging Text-To-Image Latent Diffusion Models for Metric Depth Estimation

arXiv 2411.10886 ↗First posted 16 Nov 2024cs.CV cs.AI cs.GRcited by 0

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

Recovering metric depth from a single image remains a fundamental challenge in computer vision, requiring both scene understanding and accurate scaling. While deep learning has advanced monocular depth estimation, current models often struggle with unfamiliar scenes and layouts, particularly in zero-shot scenarios and when predicting scale-ergodic metric depth. We present MetricGold, a novel approach that harnesses generative diffusion model's rich priors to improve metric depth estimation. Building upon recent advances in MariGold, DDVM and Depth Anything V2 respectively, our method combines latent diffusion, log-scaled metric depth representation, and synthetic data training. MetricGold achieves efficient training on a single RTX 3090 within two days using photo-realistic synthetic data from HyperSIM, VirtualKitti, and TartanAir. Our experiments demonstrate robust generalization across diverse datasets, producing sharper and higher quality metric depth estimates compared to existing approaches.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 14 of its 16 references that are in the map. First posted on arXiv 16 Nov 2024.

Builds on

14 of 16 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Zero-Shot Metric Depth with a Field-of-View Conditioned Diffusion Model20 Dec
  2. 2023Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation4 Dec

Other references · 12

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  2. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  3. 2021Vision Transformers for Dense Predictioncited by 1,803
  4. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,495
  5. 2023Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inferencecited by 846
  6. 2020Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understandingcited by 604
  7. 2020TartanAir: A Dataset to Push the Limits of Visual SLAMcited by 506
  8. 2023Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Imagecited by 365
  9. 2020Virtual KITTI 2cited by 353
  10. 2021Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scanscited by 336
  11. 2023Towards Zero-Shot Scale-Aware Monocular Depth Estimationcited by 101
  12. 2023VA-DepthNet: A Variational Approach to Single Image Depth Predictioncited by 54

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024SharpDepth: Sharpening Metric Depth Predictions Using Diffusion Distillation
  2. 2026MetricAnything: Scaling Metric Depth Pretraining with Noisy Heterogeneous Sources
  3. 2024GRIN: Zero-Shot Metric Depth with Pixel-Level Diffusion
  4. 2025GeoDiff: Geometry-Guided Diffusion for Metric Depth Estimation
  5. 2026Marigold V2: Revisiting Diffusion Transformers for Monocular Depth Estimation
  6. 2025Repurposing Marigold for Zero-Shot Metric Depth Estimation via Defocus Blur Cues
  7. 2024Diffusion Models for Monocular Depth Estimation: Overcoming Challenging Conditions
  8. 2024Iris: Integrating Language into Diffusion-based Monocular Depth Estimation