# Multimodal Scale Consistency and Awareness for Monocular Self-Supervised Depth Estimation

arXiv: https://arxiv.org/abs/2103.02451 · HTML: https://mlsanity.com/paper/2103.02451/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 45 references that are in the map.
- First posted on arXiv 3 Mar 2021.

## Builds on (most cited first, 30 of 36)

- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [Unsupervised Monocular Depth Estimation with Left-Right Consistency](https://mlsanity.com/paper/1609.03677/) (2016)
- [Unsupervised Learning of Depth and Ego-Motion from Video](https://mlsanity.com/paper/1704.07813/) (2017)
- [Deep Ordinal Regression Network for Monocular Depth Estimation](https://mlsanity.com/paper/1806.02446/) (2018)
- [3D Packing for Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/1905.02693/) (2019)
- [Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue](https://mlsanity.com/paper/1603.04992/) (2016)
- [GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Pose](https://mlsanity.com/paper/1803.02276/) (2018)
- [Sparsity Invariant CNNs](https://mlsanity.com/paper/1708.06500/) (2017)
- [Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Video](https://mlsanity.com/paper/1908.10553/) (2019)
- [Enforcing geometric constraints of virtual normal for depth prediction](https://mlsanity.com/paper/1907.12209/) (2019)
- [Competitive Collaboration: Joint Unsupervised Learning of Depth, Camera Motion, Optical Flow and Motion Segmentation](https://mlsanity.com/paper/1805.09806/) (2018)
- [Unsupervised Learning of Depth and Ego-Motion from Monocular Video Using 3D Geometric Constraints](https://mlsanity.com/paper/1802.05522/) (2018)
- [BA-Net: Dense Bundle Adjustment Network](https://mlsanity.com/paper/1806.04807/) (2018)
- [Depth from Videos in the Wild: Unsupervised Monocular Depth Learning from Unknown Cameras](https://mlsanity.com/paper/1904.04998/) (2019)
- [Depth Prediction Without the Sensors: Leveraging Structure for Unsupervised Learning from Monocular Videos](https://mlsanity.com/paper/1811.06152/) (2018)
- [Self-Supervised Monocular Depth Estimation: Solving the Dynamic Object Problem by Semantic Guidance](https://mlsanity.com/paper/2007.06936/) (2020)
- [Pattern-Affinitive Propagation across Depth, Surface Normal and Semantic Segmentation](https://mlsanity.com/paper/1906.03525/) (2019)
- [Measuring the tendency of CNNs to Learn Surface Statistical Regularities](https://mlsanity.com/paper/1711.11561/) (2017)
- [DeepTAM: Deep Tracking and Mapping](https://mlsanity.com/paper/1808.01900/) (2018)
- [Learning Monocular Depth by Distilling Cross-domain Stereo Networks](https://mlsanity.com/paper/1808.06586/) (2018)
- [DepthTransfer: Depth Extraction from Video Using Non-parametric Sampling](https://mlsanity.com/paper/2001.00987/) (2019)
- [Toward Hierarchical Self-Supervised Monocular Absolute Depth Estimation for Autonomous Driving Applications](https://mlsanity.com/paper/2004.05560/) (2020)
- [Deep attention-based classification network for robust depth prediction](https://mlsanity.com/paper/1807.03959/) (2018)
- [Learning to Localize Using a LiDAR Intensity Map](https://mlsanity.com/paper/2012.10902/) (2020)
- [Robust Semi-Supervised Monocular Depth Estimation with Reprojected Distances](https://mlsanity.com/paper/1910.01765/) (2019)
- [Monocular Depth Estimation with Hierarchical Fusion of Dilated CNNs and Soft-Weighted-Sum Inference](https://mlsanity.com/paper/1708.02287/) (2017)
- [Deep Robust Single Image Depth Estimation Neural Network Using Scene Understanding](https://mlsanity.com/paper/1906.03279/) (2019)

## Built on by (most cited first, 16 of 16)

- [Towards Zero-Shot Scale-Aware Monocular Depth Estimation](https://mlsanity.com/paper/2306.17253/) (2023)
- [Towards Scale-Aware, Robust, and Generalizable Unsupervised Monocular Depth Estimation by Integrating IMU Motion Dynamics](https://mlsanity.com/paper/2207.04680/) (2022)
- [Transformers in Self-Supervised Monocular Depth Estimation with Unknown Camera Intrinsics](https://mlsanity.com/paper/2202.03131/) (2022)
- [Vision-Language Embodiment for Monocular Depth Estimation](https://mlsanity.com/paper/2503.16535/) (2025)
- [Learnable Differencing Center for Nighttime Depth Perception](https://mlsanity.com/paper/2306.14538/) (2023)
- [Embodiment: Self-Supervised Depth Estimation Based on Camera Models](https://mlsanity.com/paper/2408.01565/) (2024)
- [Adversarial Attacks on Monocular Pose Estimation](https://mlsanity.com/paper/2207.07032/) (2022)
- [Image Masking for Robust Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/2210.02357/) (2022)
- [Continual Learning of Unsupervised Monocular Depth from Videos](https://mlsanity.com/paper/2311.02393/) (2023)
- [Camera Height Doesn't Change: Unsupervised Training for Metric Monocular Road-Scene Depth Estimation](https://mlsanity.com/paper/2312.04530/) (2023)
- [AI-Driven Road Maintenance Inspection v2: Reducing Data Dependency & Quantifying Road Damage](https://mlsanity.com/paper/2210.03570/) (2022)
- [Do More With What You Have: Transferring Depth-Scale from Labeled to Unlabeled Domains](https://mlsanity.com/paper/2303.07662/) (2023)
- [Pose Constraints for Consistent Self-supervised Monocular Depth and Ego-motion](https://mlsanity.com/paper/2304.08916/) (2023)
- [Transformers in Unsupervised Structure-from-Motion](https://mlsanity.com/paper/2312.10529/) (2023)
- [Multiple Prior Representation Learning for Self-Supervised Monocular Depth Estimation via Hybrid Transformer](https://mlsanity.com/paper/2406.08928/) (2024)
- [Depth3DLane: Fusing Monocular 3D Lane Detection with Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/2507.13857/) (2025)

## Abstract (arXiv)

Dense depth estimation is essential to scene-understanding for autonomous driving. However, recent self-supervised approaches on monocular videos suffer from scale-inconsistency across long sequences. Utilizing data from the ubiquitously copresent global positioning systems (GPS), we tackle this challenge by proposing a dynamically-weighted GPS-to-Scale (g2s) loss to complement the appearance-based losses. We emphasize that the GPS is needed only during the multimodal training, and not at inference. The relative distance between frames captured through the GPS provides a scale signal that is independent of the camera setup and scene distribution, resulting in richer learned feature representations. Through extensive evaluation on multiple datasets, we demonstrate scale-consistent and -aware depth estimation during inference, improving the performance even when training with low-frequency GPS data.