# BetterDepth: Plug-and-Play Diffusion Refiner for Zero-Shot Monocular Depth Estimation

arXiv: https://arxiv.org/abs/2407.17952 · HTML: https://mlsanity.com/paper/2407.17952/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 46 of its 56 references that are in the map.
- First posted on arXiv 25 Jul 2024.

## Builds on (most cited first, 30 of 46)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [Image Super-Resolution via Iterative Refinement](https://mlsanity.com/paper/2104.07636/) (2021)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [RePaint: Inpainting using Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2201.09865/) (2022)
- [Depth Anything V2](https://mlsanity.com/paper/2406.09414/) (2024)
- [Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data](https://mlsanity.com/paper/2401.10891/) (2024)
- [Palette: Image-to-Image Diffusion Models](https://mlsanity.com/paper/2111.05826/) (2021)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference](https://mlsanity.com/paper/2310.04378/) (2023)
- [MegaDepth: Learning Single-View Depth Prediction from Internet Photos](https://mlsanity.com/paper/1804.00607/) (2018)
- [Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding](https://mlsanity.com/paper/2011.02523/) (2020)
- [Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving](https://mlsanity.com/paper/1812.07179/) (2018)
- [SRDiff: Single Image Super-Resolution with Diffusion Probabilistic Models](https://mlsanity.com/paper/2104.14951/) (2021)
- [Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimation](https://mlsanity.com/paper/2404.15506/) (2024)
- [UniDepth: Universal Monocular Metric Depth Estimation](https://mlsanity.com/paper/2403.18913/) (2024)
- [Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Image](https://mlsanity.com/paper/2307.10984/) (2023)
- [Virtual KITTI 2](https://mlsanity.com/paper/2001.10773/) (2020)
- [Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scans](https://mlsanity.com/paper/2110.04994/) (2021)
- [Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation](https://mlsanity.com/paper/2312.02145/) (2023)
- [GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Image](https://mlsanity.com/paper/2403.12013/) (2024)
- [Unleashing Text-to-Image Diffusion Models for Visual Perception](https://mlsanity.com/paper/2303.02153/) (2023)
- [Deblurring via Stochastic Refinement](https://mlsanity.com/paper/2112.02475/) (2021)
- [DIODE: A Dense Indoor and Outdoor DEpth Dataset](https://mlsanity.com/paper/1908.00463/) (2019)
- [Learning to Recover 3D Scene Shape from a Single Image](https://mlsanity.com/paper/2012.09365/) (2020)

## Built on by (most cited first, 28 of 28)

- [Marigold: Affordable Adaptation of Diffusion-Based Image Generators for Image Analysis](https://mlsanity.com/paper/2505.09358/) (2025)
- [Pixel-Perfect Depth with Semantics-Prompted Diffusion Transformers](https://mlsanity.com/paper/2510.07316/) (2025)
- [Marigold-DC: Zero-Shot Monocular Depth Completion with Guided Diffusion](https://mlsanity.com/paper/2412.13389/) (2024)
- [Video Depth without Video Models](https://mlsanity.com/paper/2411.19189/) (2024)
- [SharpDepth: Sharpening Metric Depth Predictions Using Diffusion Distillation](https://mlsanity.com/paper/2411.18229/) (2024)
- [The Fourth Monocular Depth Estimation Challenge](https://mlsanity.com/paper/2504.17787/) (2025)
- [High-Precision Dichotomous Image Segmentation via Probing Diffusion Capacity](https://mlsanity.com/paper/2410.10105/) (2024)
- [StereoDiff: Stereo-Diffusion Synergy for Video Depth Estimation](https://mlsanity.com/paper/2506.20756/) (2025)
- [FiffDepth: Feed-forward Transformation of Diffusion-Based Generators for Detailed Depth Estimation](https://mlsanity.com/paper/2412.00671/) (2024)
- [SDMatte: Grafting Diffusion Models for Interactive Matting](https://mlsanity.com/paper/2508.00443/) (2025)
- [Denoise to Track: Harnessing Video Diffusion Priors for Robust Correspondence](https://mlsanity.com/paper/2512.04619/) (2025)
- [Stable-Sim2Real: Exploring Simulation of Real-Captured 3D Data with Two-Stage Depth Diffusion](https://mlsanity.com/paper/2507.23483/) (2025)
- [More Than Generation: Unifying Generation and Depth Estimation via Text-to-Image Diffusion Models](https://mlsanity.com/paper/2510.23574/) (2025)
- [StableDPT: Temporal Stable Monocular Video Depth Estimation](https://mlsanity.com/paper/2601.02793/) (2026)
- [Guardians of the Hair: Rescuing Soft Boundaries in Depth, Stereo, and Novel Views](https://mlsanity.com/paper/2601.03362/) (2026)
- [Pixel-Perfect Visual Geometry Estimation](https://mlsanity.com/paper/2601.05246/) (2026)
- [RePack then Refine: Efficient Diffusion Transformer with Vision Foundation Model](https://mlsanity.com/paper/2512.12083/) (2025)
- [ZipDepth: Bringing Lightweight Zero-Shot Monocular Depth Anywhere, on Any Device](https://mlsanity.com/paper/2607.08771/) (2026)
- [VGLD: Visually-Guided Linguistic Disambiguation for Monocular Depth Scale Recovery](https://mlsanity.com/paper/2505.02704/) (2025)
- [ReDepth Anything: Test-Time Depth Refinement via Self-Supervised Re-lighting](https://mlsanity.com/paper/2512.17908/) (2025)
- [GenDet: Painting Colored Bounding Boxes on Images via Diffusion Model for Object Detection](https://mlsanity.com/paper/2601.07273/) (2026)
- [Robust Depth Super-Resolution via Adaptive Diffusion Sampling](https://mlsanity.com/paper/2602.09510/) (2026)
- [Need for Speed: Zero-Shot Depth Completion with Single-Step Diffusion](https://mlsanity.com/paper/2603.10584/) (2026)
- [UniFixer: A Universal Reference-Guided Fixer for Diffusion-Based View Synthesis](https://mlsanity.com/paper/2605.12169/) (2026)
- [αDepth: Learning Single-Pass Soft Boundary Decomposition for Stereo Conversion](https://mlsanity.com/paper/2606.00386/) (2026)
- [Monocular Depth Estimation from a Single Image: Progress and Opportunities](https://mlsanity.com/paper/2609.01172/) (2026)
- [Guided Super-Resolution of Digital Elevation Models with Diffusion-Based Image Generators](https://mlsanity.com/paper/2609.11886/) (2026)
- [ProDyGS: Dynamic Gaussian Splatting from a Single Static Monocular Camera](https://mlsanity.com/paper/2609.32711/) (2026)

## Abstract (arXiv)

By training over large-scale datasets, zero-shot monocular depth estimation (MDE) methods show robust performance in the wild but often suffer from insufficient detail. Although recent diffusion-based MDE approaches exhibit a superior ability to extract details, they struggle in geometrically complex scenes that challenge their geometry prior, trained on less diverse 3D data. To leverage the complementary merits of both worlds, we propose BetterDepth to achieve geometrically correct affine-invariant MDE while capturing fine details. Specifically, BetterDepth is a conditional diffusion-based refiner that takes the prediction from pre-trained MDE models as depth conditioning, in which the global depth layout is well-captured, and iteratively refines details based on the input image. For the training of such a refiner, we propose global pre-alignment and local patch masking methods to ensure BetterDepth remains faithful to the depth conditioning while learning to add fine-grained scene details. With efficient training on small-scale synthetic datasets, BetterDepth achieves state-of-the-art zero-shot MDE performance on diverse public datasets and on in-the-wild scenes. Moreover, BetterDepth can improve the performance of other MDE models in a plug-and-play manner without further re-training.