# From 2D to 3D: Re-thinking Benchmarking of Monocular Depth Prediction

arXiv: https://arxiv.org/abs/2203.08122 · HTML: https://mlsanity.com/paper/2203.08122/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 60 references that are in the map.
- First posted on arXiv 15 Mar 2022.

## Builds on (most cited first, 30 of 38)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [Matterport3D: Learning from RGB-D Data in Indoor Environments](https://mlsanity.com/paper/1709.06158/) (2017)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [A Point Set Generation Network for 3D Object Reconstruction from a Single Image](https://mlsanity.com/paper/1612.00603/) (2016)
- [Deep Ordinal Regression Network for Monocular Depth Estimation](https://mlsanity.com/paper/1806.02446/) (2018)
- [AdaBins: Depth Estimation using Adaptive Bins](https://mlsanity.com/paper/2011.14141/) (2020)
- [Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecture](https://mlsanity.com/paper/1411.4734/) (2014)
- [Semantic Scene Completion from a Single Depth Image](https://mlsanity.com/paper/1611.08974/) (2016)
- [Joint 2D-3D-Semantic Data for Indoor Scene Understanding](https://mlsanity.com/paper/1702.01105/) (2017)
- [Deeper Depth Prediction with Fully Convolutional Residual Networks](https://mlsanity.com/paper/1606.00373/) (2016)
- [PCN: Point Completion Network](https://mlsanity.com/paper/1808.00671/) (2018)
- [From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimation](https://mlsanity.com/paper/1907.10326/) (2019)
- [What Do Single-view 3D Reconstruction Networks Learn?](https://mlsanity.com/paper/1905.03678/) (2019)
- [DIODE: A Dense Indoor and Outdoor DEpth Dataset](https://mlsanity.com/paper/1908.00463/) (2019)
- [Enforcing geometric constraints of virtual normal for depth prediction](https://mlsanity.com/paper/1907.12209/) (2019)
- [3D Photography using Context-aware Layered Depth Inpainting](https://mlsanity.com/paper/2004.04727/) (2020)
- [RIO: 3D Object Instance Re-Localization in Changing Indoor Environments](https://mlsanity.com/paper/1908.06109/) (2019)
- [Single-Image Depth Perception in the Wild](https://mlsanity.com/paper/1604.03901/) (2016)
- [GRNet: Gridding Residual Network for Dense Point Cloud Completion](https://mlsanity.com/paper/2006.03761/) (2020)
- [CNN-SLAM: Real-time dense monocular SLAM with learned depth prediction](https://mlsanity.com/paper/1704.03489/) (2017)
- [Revisiting Single Image Depth Estimation: Toward Higher Resolution Maps with Accurate Object Boundaries](https://mlsanity.com/paper/1803.08673/) (2018)
- [Evaluation of CNN-based Single-Image Depth Estimation Methods](https://mlsanity.com/paper/1805.01328/) (2018)
- [Morphing and Sampling Network for Dense Point Cloud Completion](https://mlsanity.com/paper/1912.00280/) (2019)
- [3D Ken Burns Effect from a Single Image](https://mlsanity.com/paper/1909.05483/) (2019)
- [Transformer-Based Attention Networks for Continuous Pixel-Wise Prediction](https://mlsanity.com/paper/2103.12091/) (2021)
- [PlaneRCNN: 3D Plane Detection and Reconstruction from a Single Image](https://mlsanity.com/paper/1812.04072/) (2018)
- [Variational Relational Point Completion Network](https://mlsanity.com/paper/2104.10154/) (2021)
- [SharpNet: Fast and Accurate Recovery of Occluding Contours in Monocular Depth Estimation](https://mlsanity.com/paper/1905.08598/) (2019)

## Built on by (most cited first, 19 of 19)

- [UniDepth: Universal Monocular Metric Depth Estimation](https://mlsanity.com/paper/2403.18913/) (2024)
- [UniDepthV2: Universal Monocular Metric Depth Estimation Made Simpler](https://mlsanity.com/paper/2502.20110/) (2025)
- [UniK3D: Universal Camera Monocular 3D Estimation](https://mlsanity.com/paper/2503.16591/) (2025)
- [CommonScenes: Generating Commonsense 3D Indoor Scenes with Scene Graph Diffusion](https://mlsanity.com/paper/2305.16283/) (2023)
- [GEDepth: Ground Embedding for Monocular Depth Estimation](https://mlsanity.com/paper/2309.09975/) (2023)
- [Kick Back & Relax: Learning to Reconstruct the World by Watching SlowTV](https://mlsanity.com/paper/2307.10713/) (2023)
- [Deconstructing Self-Supervised Monocular Reconstruction: The Design Decisions that Matter](https://mlsanity.com/paper/2208.01489/) (2022)
- [The Monocular Depth Estimation Challenge](https://mlsanity.com/paper/2211.12174/) (2022)
- [The Second Monocular Depth Estimation Challenge](https://mlsanity.com/paper/2304.07051/) (2023)
- [The Third Monocular Depth Estimation Challenge](https://mlsanity.com/paper/2404.16831/) (2024)
- [The Fourth Monocular Depth Estimation Challenge](https://mlsanity.com/paper/2504.17787/) (2025)
- [LaRI: Layered Ray Intersections for Single-view 3D Geometric Reasoning](https://mlsanity.com/paper/2504.18424/) (2025)
- [Kick Back & Relax++: Scaling Beyond Ground-Truth Depth with SlowTV & CribsTV](https://mlsanity.com/paper/2403.01569/) (2024)
- [SupeRGB-D: Zero-shot Instance Segmentation in Cluttered Indoor Environments](https://mlsanity.com/paper/2212.11922/) (2022)
- [RPG360: Robust 360 Depth Estimation with Perspective Foundation Models and Graph Optimization](https://mlsanity.com/paper/2509.23991/) (2025)
- [BaseBoostDepth: Exploiting Larger Baselines For Self-supervised Monocular Depth Estimation](https://mlsanity.com/paper/2407.20437/) (2024)
- [Refinement of Monocular Depth Maps via Multi-View Differentiable Rendering](https://mlsanity.com/paper/2410.03861/) (2024)
- [Toward A Better Understanding of Monocular Depth Evaluation](https://mlsanity.com/paper/2510.19814/) (2025)
- [Introducing a Class-Aware Metric for Monocular Depth Estimation: An Automotive Perspective](https://mlsanity.com/paper/2409.04086/) (2024)

## Abstract (arXiv)

There have been numerous recently proposed methods for monocular depth prediction (MDP) coupled with the equally rapid evolution of benchmarking tools. However, we argue that MDP is currently witnessing benchmark over-fitting and relying on metrics that are only partially helpful to gauge the usefulness of the predictions for 3D applications. This limits the design and development of novel methods that are truly aware of - and improving towards estimating - the 3D structure of the scene rather than optimizing 2D-based distances. In this work, we aim to bring structural awareness to MDP, an inherently 3D task, by exhibiting the limits of evaluation metrics towards assessing the quality of the 3D geometry. We propose a set of metrics well suited to evaluate the 3D geometry of MDP approaches and a novel indoor benchmark, RIO-D3D, crucial for the proposed evaluation methodology. Our benchmark is based on a real-world dataset featuring high-quality rendered depth maps obtained from RGB-D reconstructions. We further demonstrate this to help benchmark the closely-tied task of 3D scene completion.