# Deconstructing Self-Supervised Monocular Reconstruction: The Design Decisions that Matter

arXiv: https://arxiv.org/abs/2208.01489 · HTML: https://mlsanity.com/paper/2208.01489/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 58 references that are in the map.
- First posted on arXiv 2 Aug 2022.

## Builds on (most cited first, 30 of 45)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation](https://mlsanity.com/paper/1512.02134/) (2015)
- [Unsupervised Monocular Depth Estimation with Left-Right Consistency](https://mlsanity.com/paper/1609.03677/) (2016)
- [Unsupervised Learning of Depth and Ego-Motion from Video](https://mlsanity.com/paper/1704.07813/) (2017)
- [Direct Sparse Odometry](https://mlsanity.com/paper/1607.02565/) (2016)
- [Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecture](https://mlsanity.com/paper/1411.4734/) (2014)
- [3D Packing for Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/1905.02693/) (2019)
- [Deeper Depth Prediction with Fully Convolutional Residual Networks](https://mlsanity.com/paper/1606.00373/) (2016)
- [Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue](https://mlsanity.com/paper/1603.04992/) (2016)
- [Sparsity Invariant CNNs](https://mlsanity.com/paper/1708.06500/) (2017)
- [Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Video](https://mlsanity.com/paper/1908.10553/) (2019)
- [Billion-scale semi-supervised learning for image classification](https://mlsanity.com/paper/1905.00546/) (2019)
- [D3VO: Deep Depth, Deep Pose and Deep Uncertainty for Monocular Visual Odometry](https://mlsanity.com/paper/2003.01060/) (2020)
- [On the uncertainty of self-supervised monocular depth estimation](https://mlsanity.com/paper/2005.06209/) (2020)
- [Unsupervised Learning of Depth and Ego-Motion from Monocular Video Using 3D Geometric Constraints](https://mlsanity.com/paper/1802.05522/) (2018)
- [Unsupervised Learning of Monocular Depth Estimation and Visual Odometry with Deep Feature Reconstruction](https://mlsanity.com/paper/1803.03893/) (2018)
- [Learning Depth from Monocular Videos using Direct Methods](https://mlsanity.com/paper/1712.00175/) (2017)
- [Feature-metric Loss for Self-supervised Learning of Depth and Egomotion](https://mlsanity.com/paper/2007.10603/) (2020)
- [Evaluation of CNN-based Single-Image Depth Estimation Methods](https://mlsanity.com/paper/1805.01328/) (2018)
- [Self-Supervised Monocular Depth Hints](https://mlsanity.com/paper/1909.09051/) (2019)
- [Self-supervised Monocular Trained Depth Estimation using Self-attention and Discrete Disparity Volume](https://mlsanity.com/paper/2003.13951/) (2020)

## Built on by (most cited first, 18 of 18)

- [Depth Pro: Sharp Monocular Metric Depth in Less Than a Second](https://mlsanity.com/paper/2410.02073/) (2024)
- [Kick Back & Relax: Learning to Reconstruct the World by Watching SlowTV](https://mlsanity.com/paper/2307.10713/) (2023)
- [DepthCues: Evaluating Monocular Depth Perception in Large Vision Models](https://mlsanity.com/paper/2411.17385/) (2024)
- [The Monocular Depth Estimation Challenge](https://mlsanity.com/paper/2211.12174/) (2022)
- [The Second Monocular Depth Estimation Challenge](https://mlsanity.com/paper/2304.07051/) (2023)
- [The Third Monocular Depth Estimation Challenge](https://mlsanity.com/paper/2404.16831/) (2024)
- [Scalable Semantic 3D Mapping of Coral Reefs with Deep Learning](https://mlsanity.com/paper/2309.12804/) (2023)
- [The Fourth Monocular Depth Estimation Challenge](https://mlsanity.com/paper/2504.17787/) (2025)
- [Kick Back & Relax++: Scaling Beyond Ground-Truth Depth with SlowTV & CribsTV](https://mlsanity.com/paper/2403.01569/) (2024)
- [Position: Machine Learning Conferences Should Establish a "Refutations and Critiques" Track](https://mlsanity.com/paper/2506.19882/) (2025)
- [Equivariant Single View Pose Prediction Via Induced and Restricted Representations](https://mlsanity.com/paper/2307.03704/) (2023)
- [RPG360: Robust 360 Depth Estimation with Perspective Foundation Models and Graph Optimization](https://mlsanity.com/paper/2509.23991/) (2025)
- [UnDIVE: Generalized Underwater Video Enhancement Using Generative Priors](https://mlsanity.com/paper/2411.05886/) (2024)
- [BaseBoostDepth: Exploiting Larger Baselines For Self-supervised Monocular Depth Estimation](https://mlsanity.com/paper/2407.20437/) (2024)
- [NimbleD: Enhancing Self-supervised Monocular Depth Estimation with Pseudo-labels and Large-scale Video Pre-training](https://mlsanity.com/paper/2408.14177/) (2024)
- [HQDec: Self-Supervised Monocular Depth Estimation Based on a High-Quality Decoder](https://mlsanity.com/paper/2305.18706/) (2023)
- [Towards Sharper Object Boundaries in Self-Supervised Depth Estimation](https://mlsanity.com/paper/2509.15987/) (2025)
- [EDADepth: Enhanced Data Augmentation for Monocular Depth Estimation](https://mlsanity.com/paper/2409.06183/) (2024)

## Abstract (arXiv)

This paper presents an open and comprehensive framework to systematically evaluate state-of-the-art contributions to self-supervised monocular depth estimation. This includes pretraining, backbone, architectural design choices and loss functions. Many papers in this field claim novelty in either architecture design or loss formulation. However, simply updating the backbone of historical systems results in relative improvements of 25%, allowing them to outperform the majority of existing systems. A systematic evaluation of papers in this field was not straightforward. The need to compare like-with-like in previous papers means that longstanding errors in the evaluation protocol are ubiquitous in the field. It is likely that many papers were not only optimized for particular datasets, but also for errors in the data and evaluation criteria. To aid future research in this area, we release a modular codebase (https://github.com/jspenmar/monodepth_benchmark), allowing for easy evaluation of alternate design decisions against corrected data and evaluation criteria. We re-implement, validate and re-evaluate 16 state-of-the-art contributions and introduce a new dataset (SYNS-Patches) containing dense outdoor depth maps in a variety of both natural and urban scenes. This allows for the computation of informative metrics in complex regions such as depth boundaries.