# Deep Digging into the Generalization of Self-Supervised Monocular Depth Estimation

arXiv: https://arxiv.org/abs/2205.11083 · HTML: https://mlsanity.com/paper/2205.11083/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 49 references that are in the map.
- First posted on arXiv 23 May 2022.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [UNet++: A Nested U-Net Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1807.10165/) (2018)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [Dual Attention Network for Scene Segmentation](https://mlsanity.com/paper/1809.02983/) (2018)
- [Self-Attention Generative Adversarial Networks](https://mlsanity.com/paper/1805.08318/) (2018)
- [Unsupervised Learning of Depth and Ego-Motion from Video](https://mlsanity.com/paper/1704.07813/) (2017)
- [Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecture](https://mlsanity.com/paper/1411.4734/) (2014)
- [3D Packing for Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/1905.02693/) (2019)
- [GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Pose](https://mlsanity.com/paper/1803.02276/) (2018)
- [Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNet](https://mlsanity.com/paper/1904.00760/) (2019)
- [DeMoN: Depth and Motion Network for Learning Monocular Stereo](https://mlsanity.com/paper/1612.02401/) (2016)
- [Depth Prediction Without the Sensors: Leveraging Structure for Unsupervised Learning from Monocular Videos](https://mlsanity.com/paper/1811.06152/) (2018)
- [Self-Supervised Monocular Depth Estimation: Solving the Dynamic Object Problem by Semantic Guidance](https://mlsanity.com/paper/2007.06936/) (2020)
- [Semantically-Guided Representation Learning for Self-Supervised Monocular Depth](https://mlsanity.com/paper/2002.12319/) (2020)
- [Transformer-Based Attention Networks for Continuous Pixel-Wise Prediction](https://mlsanity.com/paper/2103.12091/) (2021)
- [HR-Depth: High Resolution Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/2012.07356/) (2020)
- [Are Convolutional Neural Networks or Transformers more like human vision?](https://mlsanity.com/paper/2105.07197/) (2021)
- [Delving Deep into the Generalization of Vision Transformers under Distribution Shifts](https://mlsanity.com/paper/2106.07617/) (2021)
- [Multi-Frame Self-Supervised Depth with Transformers](https://mlsanity.com/paper/2204.07616/) (2022)
- [A Disentangling Invertible Interpretation Network for Explaining Latent Representations](https://mlsanity.com/paper/2004.13166/) (2020)
- [Shape or Texture: Understanding Discriminative Features in CNNs](https://mlsanity.com/paper/2101.11604/) (2021)

## Built on by (most cited first, 24 of 24)

- [Depth Prompting for Sensor-Agnostic Depth Estimation](https://mlsanity.com/paper/2405.11867/) (2024)
- [Deep Learning-based Depth Estimation Methods from Monocular Image and Videos: A Comprehensive Survey](https://mlsanity.com/paper/2406.19675/) (2024)
- [MonoDiffusion: Self-Supervised Monocular Depth Estimation Using Diffusion Model](https://mlsanity.com/paper/2311.07198/) (2023)
- [From-Ground-To-Objects: Coarse-to-Fine Self-supervised Monocular Depth Estimation of Dynamic Objects with Ground Contact Prior](https://mlsanity.com/paper/2312.10118/) (2023)
- [IDOL: Unified Dual-Modal Latent Diffusion for Human-Centric Joint Video-Depth Generation](https://mlsanity.com/paper/2407.10937/) (2024)
- [Mono-ViFI: A Unified Learning Framework for Self-supervised Single- and Multi-frame Monocular Depth Estimation](https://mlsanity.com/paper/2407.14126/) (2024)
- [FisheyeDepth: A Real Scale Self-Supervised Depth Estimation Model for Fisheye Camera](https://mlsanity.com/paper/2409.15054/) (2024)
- [ProDepth: Boosting Self-Supervised Multi-Frame Monocular Depth with Probabilistic Fusion](https://mlsanity.com/paper/2407.09303/) (2024)
- [Hybrid-grained Feature Aggregation with Coarse-to-fine Language Guidance for Self-supervised Monocular Depth Estimation](https://mlsanity.com/paper/2510.09320/) (2025)
- [Improving Domain Generalization in Self-supervised Monocular Depth Estimation via Stabilized Adversarial Training](https://mlsanity.com/paper/2411.02149/) (2024)
- [A Study on the Generality of Neural Network Structures for Monocular Depth Estimation](https://mlsanity.com/paper/2301.03169/) (2023)
- [Self-Supervised Monocular Depth Estimation in the Dark: Towards Data Distribution Compensation](https://mlsanity.com/paper/2404.13854/) (2024)
- [MonoPP: Metric-Scaled Self-Supervised Monocular Depth Estimation by Planar-Parallax Geometry in Automotive Applications](https://mlsanity.com/paper/2411.19717/) (2024)
- [On Robust Cross-View Consistency in Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/2209.08747/) (2022)
- [EndoDepthL: Lightweight Endoscopic Monocular Depth Estimation with CNN-Transformer](https://mlsanity.com/paper/2308.02716/) (2023)
- [Channel-Aware Distillation Transformer for Depth Estimation on Nano Drones](https://mlsanity.com/paper/2303.10386/) (2023)
- [Generalizing monocular colonoscopy image depth estimation by uncertainty-based global and local fusion network](https://mlsanity.com/paper/2409.15006/) (2024)
- [Federated Self-Supervised Learning of Monocular Depth Estimators for Autonomous Vehicles](https://mlsanity.com/paper/2310.04837/) (2023)
- [Transformers in Unsupervised Structure-from-Motion](https://mlsanity.com/paper/2312.10529/) (2023)
- [Back2Color: Domain-Adaptive Synthetic-to-Real Monocular Depth Estimation for Dynamic Traffic Scenes](https://mlsanity.com/paper/2406.07741/) (2024)
- [Multiple Prior Representation Learning for Self-Supervised Monocular Depth Estimation via Hybrid Transformer](https://mlsanity.com/paper/2406.08928/) (2024)
- [Bridging Geometric and Semantic Foundation Models for Generalized Monocular Depth Estimation](https://mlsanity.com/paper/2505.23400/) (2025)
- [DepthDark: Robust Monocular Depth Estimation for Low-Light Environments](https://mlsanity.com/paper/2507.18243/) (2025)
- [Monocular Depth Estimation via Neural Network with Learnable Algebraic Group and Ring Structures](https://mlsanity.com/paper/2604.24328/) (2026)

## Abstract (arXiv)

Self-supervised monocular depth estimation has been widely studied recently. Most of the work has focused on improving performance on benchmark datasets, such as KITTI, but has offered a few experiments on generalization performance. In this paper, we investigate the backbone networks (e.g. CNNs, Transformers, and CNN-Transformer hybrid models) toward the generalization of monocular depth estimation. We first evaluate state-of-the-art models on diverse public datasets, which have never been seen during the network training. Next, we investigate the effects of texture-biased and shape-biased representations using the various texture-shifted datasets that we generated. We observe that Transformers exhibit a strong shape bias and CNNs do a strong texture-bias. We also find that shape-biased models show better generalization performance for monocular depth estimation compared to texture-biased models. Based on these observations, we newly design a CNN-Transformer hybrid network with a multi-level adaptive feature fusion module, called MonoFormer. The design intuition behind MonoFormer is to increase shape bias by employing Transformers while compensating for the weak locality bias of Transformers by adaptively fusing multi-level representations. Extensive experiments show that the proposed method achieves state-of-the-art performance with various public datasets. Our method also shows the best generalization ability among the competitive methods.