# PatchRefiner: Leveraging Synthetic Data for Real-Domain High-Resolution Monocular Metric Depth Estimation

arXiv: https://arxiv.org/abs/2406.06679 · HTML: https://mlsanity.com/paper/2406.06679/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 49 of its 71 references that are in the map.
- First posted on arXiv 10 Jun 2024.

## Builds on (most cited first, 30 of 49)

- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data](https://mlsanity.com/paper/2401.10891/) (2024)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenes](https://mlsanity.com/paper/2308.11417/) (2023)
- [Deep Ordinal Regression Network for Monocular Depth Estimation](https://mlsanity.com/paper/1806.02446/) (2018)
- [ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depth](https://mlsanity.com/paper/2302.12288/) (2023)
- [AdaBins: Depth Estimation using Adaptive Bins](https://mlsanity.com/paper/2011.14141/) (2020)
- [Learning Continuous Image Representation with Local Implicit Image Function](https://mlsanity.com/paper/2012.09161/) (2020)
- [Noise2Noise: Learning Image Restoration without Clean Data](https://mlsanity.com/paper/1803.04189/) (2018)
- [Bidirectional Learning for Domain Adaptation of Semantic Segmentation](https://mlsanity.com/paper/1904.10620/) (2019)
- [A Brief Review of Domain Adaptation](https://mlsanity.com/paper/2010.03978/) (2020)
- [DeepMVS: Learning Multi-view Stereopsis](https://mlsanity.com/paper/1804.00650/) (2018)
- [Unleashing Text-to-Image Diffusion Models for Visual Perception](https://mlsanity.com/paper/2303.02153/) (2023)
- [Progressive Feature Alignment for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1811.08585/) (2018)
- [Single-Image Depth Perception in the Wild](https://mlsanity.com/paper/1604.03901/) (2016)
- [BinsFormer: Revisiting Adaptive Bins for Monocular Depth Estimation](https://mlsanity.com/paper/2204.00987/) (2022)
- [Asymmetric Tri-training for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1702.08400/) (2017)
- [Evaluation of CNN-based Single-Image Depth Estimation Methods](https://mlsanity.com/paper/1805.01328/) (2018)
- [SMD-Nets: Stereo Mixture Density Networks](https://mlsanity.com/paper/2104.03866/) (2021)
- [DepthFormer: Exploiting Long-Range Correlation and Local Information for Accurate Monocular Depth Estimation](https://mlsanity.com/paper/2203.14211/) (2022)
- [CrDoCo: Pixel-level Domain Transfer with Cross-Domain Consistency](https://mlsanity.com/paper/2001.03182/) (2020)
- [Boosting Monocular Depth Estimation Models to High-Resolution via Content-Adaptive Multi-Resolution Merging](https://mlsanity.com/paper/2105.14021/) (2021)
- [MM-TTA: Multi-Modal Test-Time Adaptation for 3D Semantic Segmentation](https://mlsanity.com/paper/2204.12667/) (2022)
- [iDisc: Internal Discretization for Monocular Depth Estimation](https://mlsanity.com/paper/2304.06334/) (2023)
- [Discrete Cosine Transform Network for Guided Depth Map Super-Resolution](https://mlsanity.com/paper/2104.06977/) (2021)
- [T2Net: Synthetic-to-Realistic Translation for Solving Single-Image Depth Estimation Tasks](https://mlsanity.com/paper/1808.01454/) (2018)
- [SimPLE: Similar Pseudo Label Exploitation for Semi-Supervised Classification](https://mlsanity.com/paper/2103.16725/) (2021)

## Built on by (most cited first, 16 of 16)

- [Marigold: Affordable Adaptation of Diffusion-Based Image Generators for Image Analysis](https://mlsanity.com/paper/2505.09358/) (2025)
- [Prompting Depth Anything for 4K Resolution Accurate Metric Depth Estimation](https://mlsanity.com/paper/2412.14015/) (2024)
- [Pixel-Perfect Depth with Semantics-Prompted Diffusion Transformers](https://mlsanity.com/paper/2510.07316/) (2025)
- [Video Depth without Video Models](https://mlsanity.com/paper/2411.19189/) (2024)
- [SharpDepth: Sharpening Metric Depth Predictions Using Diffusion Distillation](https://mlsanity.com/paper/2411.18229/) (2024)
- [PatchRefiner V2: Fast and Lightweight Real-Domain High-Resolution Metric Depth Estimation](https://mlsanity.com/paper/2501.01121/) (2025)
- [One Look is Enough: Seamless Patchwise Refinement for Zero-Shot Monocular Depth Estimation on High-Resolution Images](https://mlsanity.com/paper/2503.22351/) (2025)
- [Pixel-Perfect Visual Geometry Estimation](https://mlsanity.com/paper/2601.05246/) (2026)
- [EgoPoseFormer v2: Accurate Egocentric Human Motion Estimation for AR/VR](https://mlsanity.com/paper/2603.04090/) (2026)
- [UniLiPs: Unified LiDAR Pseudo-Labeling with Geometry-Grounded Dynamic Scene Decomposition](https://mlsanity.com/paper/2601.05105/) (2026)
- [Any Resolution Any Geometry: From Multi-View To Multi-Patch](https://mlsanity.com/paper/2603.03026/) (2026)
- [2K Retrofit: Entropy-Guided Efficient Sparse Refinement for High-Resolution 3D Geometry Prediction](https://mlsanity.com/paper/2603.19964/) (2026)
- [PXDepth: Pixel-Space Modeling for Structure Preserving Monocular Depth Estimation](https://mlsanity.com/paper/2608.16984/) (2026)
- [Survey on Monocular Metric Depth Estimation](https://mlsanity.com/paper/2501.11841/) (2025)
- [Toward Real-World High-Precision Image Matting and Segmentation](https://mlsanity.com/paper/2601.12080/) (2026)
- [Learning Field Reconstruction from Incomplete Data by Globally Correcting Local Estimates](https://mlsanity.com/paper/2610.05375/) (2026)

## Abstract (arXiv)

This paper introduces PatchRefiner, an advanced framework for metric single image depth estimation aimed at high-resolution real-domain inputs. While depth estimation is crucial for applications such as autonomous driving, 3D generative modeling, and 3D reconstruction, achieving accurate high-resolution depth in real-world scenarios is challenging due to the constraints of existing architectures and the scarcity of detailed real-world depth data. PatchRefiner adopts a tile-based methodology, reconceptualizing high-resolution depth estimation as a refinement process, which results in notable performance enhancements. Utilizing a pseudo-labeling strategy that leverages synthetic data, PatchRefiner incorporates a Detail and Scale Disentangling (DSD) loss to enhance detail capture while maintaining scale accuracy, thus facilitating the effective transfer of knowledge from synthetic to real-world data. Our extensive evaluations demonstrate PatchRefiner's superior performance, significantly outperforming existing benchmarks on the Unreal4KStereo dataset by 18.1% in terms of the root mean squared error (RMSE) and showing marked improvements in detail accuracy and consistent scale estimation on diverse real-world datasets like CityScape, ScanNet++, and ETH3D.