# Zero-Shot Metric Depth with a Field-of-View Conditioned Diffusion Model

arXiv: https://arxiv.org/abs/2312.13252 · HTML: https://mlsanity.com/paper/2312.13252/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models.
- Builds on 47 of its 59 references that are in the map.
- First posted on arXiv 20 Dec 2023.

## Placed in

- Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/)

## Builds on (most cited first, 30 of 47)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [Palette: Image-to-Image Diffusion Models](https://mlsanity.com/paper/2111.05826/) (2021)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [Cascaded Diffusion Models for High Fidelity Image Generation](https://mlsanity.com/paper/2106.15282/) (2021)
- [Deep Ordinal Regression Network for Monocular Depth Estimation](https://mlsanity.com/paper/1806.02446/) (2018)
- [ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depth](https://mlsanity.com/paper/2302.12288/) (2023)
- [Taskonomy: Disentangling Task Transfer Learning](https://mlsanity.com/paper/1804.08328/) (2018)
- [Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding](https://mlsanity.com/paper/2011.02523/) (2020)
- [AdaBins: Depth Estimation using Adaptive Bins](https://mlsanity.com/paper/2011.14141/) (2020)
- [TartanAir: A Dataset to Push the Limits of Visual SLAM](https://mlsanity.com/paper/2003.14338/) (2020)
- [Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecture](https://mlsanity.com/paper/1411.4734/) (2014)
- [3D Packing for Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/1905.02693/) (2019)
- [Deeper Depth Prediction with Fully Convolutional Residual Networks](https://mlsanity.com/paper/1606.00373/) (2016)
- [From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimation](https://mlsanity.com/paper/1907.10326/) (2019)
- [Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Image](https://mlsanity.com/paper/2307.10984/) (2023)
- [DIODE: A Dense Indoor and Outdoor DEpth Dataset](https://mlsanity.com/paper/1908.00463/) (2019)
- [BinsFormer: Revisiting Adaptive Bins for Monocular Depth Estimation](https://mlsanity.com/paper/2204.00987/) (2022)
- [NeW CRFs: Neural Window Fully-connected CRFs for Monocular Depth Estimation](https://mlsanity.com/paper/2203.01502/) (2022)
- [DDP: Diffusion Model for Dense Visual Prediction](https://mlsanity.com/paper/2303.17559/) (2023)

## Built on by (most cited first, 29 of 29)

- [Depth Pro: Sharp Monocular Metric Depth in Less Than a Second](https://mlsanity.com/paper/2410.02073/) (2024)
- [TRAM: Global Trajectory and Motion of 3D Humans from in-the-wild Videos](https://mlsanity.com/paper/2403.17346/) (2024)
- [DepthFM: Fast Monocular Depth Estimation with Flow Matching](https://mlsanity.com/paper/2403.13788/) (2024)
- [Controlling Space and Time with Diffusion Models](https://mlsanity.com/paper/2407.07860/) (2024)
- [Gen3DSR: Generalizable 3D Scene Reconstruction via Divide and Conquer from a Single View](https://mlsanity.com/paper/2404.03421/) (2024)
- [Depth Any Video with Scalable Synthetic Data](https://mlsanity.com/paper/2410.10815/) (2024)
- [Pixel-Perfect Depth with Semantics-Prompted Diffusion Transformers](https://mlsanity.com/paper/2510.07316/) (2025)
- [The 3D-PC: a benchmark for visual perspective taking in humans and machines](https://mlsanity.com/paper/2406.04138/) (2024)
- [ScaleDepth: Decomposing Metric Depth Estimation into Scale Prediction and Relative Depth Estimation](https://mlsanity.com/paper/2407.08187/) (2024)
- [SwinMTL: A Shared Architecture for Simultaneous Depth Estimation and Semantic Segmentation from Monocular Camera Images](https://mlsanity.com/paper/2403.10662/) (2024)
- [A Simple Approach to Unifying Diffusion-based Conditional Generation](https://mlsanity.com/paper/2410.11439/) (2024)
- [PrimeDepth: Efficient Monocular Depth Estimation with a Stable Diffusion Preimage](https://mlsanity.com/paper/2409.09144/) (2024)
- [Zero-Shot Novel View and Depth Synthesis with Multi-View Geometric Diffusion](https://mlsanity.com/paper/2501.18804/) (2025)
- [High-Resolution Frame Interpolation with Patch-based Cascaded Diffusion](https://mlsanity.com/paper/2410.11838/) (2024)
- [GRIN: Zero-Shot Metric Depth with Pixel-Level Diffusion](https://mlsanity.com/paper/2409.09896/) (2024)
- [Spatially Aware World Action Model via Geometric Latent Diffusion](https://mlsanity.com/paper/2609.02531/) (2026)
- [Rethinking Iterative Stereo Matching from Diffusion Bridge Model Perspective](https://mlsanity.com/paper/2404.09051/) (2024)
- [Generative Refocusing: Flexible Defocus Control from a Single Image](https://mlsanity.com/paper/2512.16923/) (2025)
- [Repurposing Marigold for Zero-Shot Metric Depth Estimation via Defocus Blur Cues](https://mlsanity.com/paper/2505.17358/) (2025)
- [NimbleD: Enhancing Self-supervised Monocular Depth Estimation with Pseudo-labels and Large-scale Video Pre-training](https://mlsanity.com/paper/2408.14177/) (2024)
- [Pixel-Perfect Visual Geometry Estimation](https://mlsanity.com/paper/2601.05246/) (2026)
- [Diffusion-Occ: 3D Point Cloud Completion via Occupancy Diffusion](https://mlsanity.com/paper/2408.14846/) (2024)
- [Depth-Aware Scoring and Hierarchical Alignment for Multiple Object Tracking](https://mlsanity.com/paper/2506.00774/) (2025)
- [Introducing a Class-Aware Metric for Monocular Depth Estimation: An Automotive Perspective](https://mlsanity.com/paper/2409.04086/) (2024)
- [MetricGold: Leveraging Text-To-Image Latent Diffusion Models for Metric Depth Estimation](https://mlsanity.com/paper/2411.10886/) (2024)
- [Survey on Monocular Metric Depth Estimation](https://mlsanity.com/paper/2501.11841/) (2025)
- [Learn Your Scales: Towards Scale-Consistent Generative Novel View Synthesis](https://mlsanity.com/paper/2503.15412/) (2025)
- [Monocular Depth Estimation from a Single Image: Progress and Opportunities](https://mlsanity.com/paper/2609.01172/) (2026)
- [Zero-Shot Novel Depth Synthesis Using 3D Foundation Models Scene Representations](https://mlsanity.com/paper/2609.04174/) (2026)

## Abstract (arXiv)

While methods for monocular depth estimation have made significant strides on standard benchmarks, zero-shot metric depth estimation remains unsolved. Challenges include the joint modeling of indoor and outdoor scenes, which often exhibit significantly different distributions of RGB and depth, and the depth-scale ambiguity due to unknown camera intrinsics. Recent work has proposed specialized multi-head architectures for jointly modeling indoor and outdoor scenes. In contrast, we advocate a generic, task-agnostic diffusion model, with several advancements such as log-scale depth parameterization to enable joint modeling of indoor and outdoor scenes, conditioning on the field-of-view (FOV) to handle scale ambiguity and synthetically augmenting FOV during training to generalize beyond the limited camera intrinsics in training datasets. Furthermore, by employing a more diverse training mixture than is common, and an efficient diffusion parameterization, our method, DMD (Diffusion for Metric Depth) achieves a 25% reduction in relative error (REL) on zero-shot indoor and 33% reduction on zero-shot outdoor datasets over the current SOTA using only a small number of denoising steps. For an overview see https://diffusion-vision.github.io/dmd