mlsanityData to 7 Oct 2026

Multimodal Scale Consistency and Awareness for Monocular Self-Supervised Depth Estimation

arXiv 2103.02451 ↗First posted 3 Mar 2021cs.CV cs.LG cs.ROcited by 16

Not on the map. Why?

Abstract (arXiv)

Dense depth estimation is essential to scene-understanding for autonomous driving. However, recent self-supervised approaches on monocular videos suffer from scale-inconsistency across long sequences. Utilizing data from the ubiquitously copresent global positioning systems (GPS), we tackle this challenge by proposing a dynamically-weighted GPS-to-Scale (g2s) loss to complement the appearance-based losses. We emphasize that the GPS is needed only during the multimodal training, and not at inference. The relative distance between frames captured through the GPS provides a scale signal that is independent of the camera setup and scene distribution, resulting in richer learned feature representations. Through extensive evaluation on multiple datasets, we demonstrate scale-consistent and -aware depth estimation during inference, improving the performance even when training with low-frequency GPS data.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 36 of its 45 references that are in the map. First posted on arXiv 3 Mar 2021.

Builds on

36 of 45 references.

References · 36

  1. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  2. 2020Shortcut Learning in Deep Neural Networkscited by 1,928
  3. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,264
  4. 2015ORB-SLAM: a Versatile and Accurate Monocular SLAM Systemcited by 983
  5. 2016Unsupervised Monocular Depth Estimation with Left-Right Consistencycited by 819
  6. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 778
  7. 2018Deep Ordinal Regression Network for Monocular Depth Estimationcited by 721
  8. 20193D Packing for Self-Supervised Monocular Depth Estimationcited by 437
  9. 2016Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescuecited by 368
  10. 2018GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Posecited by 341
  11. 2017Sparsity Invariant CNNscited by 325
  12. 2019Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Videocited by 244
24 more
  1. 2019Enforcing geometric constraints of virtual normal for depth predictioncited by 233
  2. 2018Competitive Collaboration: Joint Unsupervised Learning of Depth, Camera Motion, Optical Flow and Motion Segmentationcited by 222
  3. 2018Unsupervised Learning of Depth and Ego-Motion from Monocular Video Using 3D Geometric Constraintscited by 179
  4. 2018BA-Net: Dense Bundle Adjustment Networkcited by 173
  5. 2019Depth from Videos in the Wild: Unsupervised Monocular Depth Learning from Unknown Camerascited by 170
  6. 2018Depth Prediction Without the Sensors: Leveraging Structure for Unsupervised Learning from Monocular Videoscited by 155
  7. 2020Self-Supervised Monocular Depth Estimation: Solving the Dynamic Object Problem by Semantic Guidancecited by 148
  8. 2019Pattern-Affinitive Propagation across Depth, Surface Normal and Semantic Segmentationcited by 141
  9. 2017Measuring the tendency of CNNs to Learn Surface Statistical Regularitiescited by 87
  10. 2018DeepTAM: Deep Tracking and Mappingcited by 87
  11. 2018Learning Monocular Depth by Distilling Cross-domain Stereo Networkscited by 64
  12. 2019DepthTransfer: Depth Extraction from Video Using Non-parametric Samplingcited by 60
  13. 2020Toward Hierarchical Self-Supervised Monocular Absolute Depth Estimation for Autonomous Driving Applicationscited by 40
  14. 2018Deep attention-based classification network for robust depth predictioncited by 36
  15. 2020Learning to Localize Using a LiDAR Intensity Mapcited by 28
  16. 2019Robust Semi-Supervised Monocular Depth Estimation with Reprojected Distancescited by 27
  17. 2017Monocular Depth Estimation with Hierarchical Fusion of Dilated CNNs and Soft-Weighted-Sum Inferencecited by 24
  18. 2019Deep Robust Single Image Depth Estimation Neural Network Using Scene Understandingcited by 15
  19. 2019MultiDepth: Single-Image Depth Estimation via Multi-Task Regression and Classificationcited by 13
  20. 2018Pixel-wise Attentional Gating for Parsimonious Pixel Labelingcited by 12
  21. 2019SDNet: Semantically Guided Depth Estimation Networkcited by 11
  22. 2019Learn Stereo, Infer Mono: Siamese Networks for Self-Supervised, Monocular, Depth Estimationcited by 6
  23. 2018An Experimental Study on Relative and Absolute Pose Graph Fusion for Vehicle Localizationcited by 3
  24. 2020Crowdsourced 3D Mapping: A Combined Multi-View Geometry and Self-Supervised Learning Approachcited by 3

Built on by

Cited by year

Citing papers per year: 2022: 5; 2023: 7; 2024: 2; 2025: 2.

16 papers.

Sort:

  1. 2023Towards Zero-Shot Scale-Aware Monocular Depth Estimationcited by 100
  2. 2022Towards Scale-Aware, Robust, and Generalizable Unsupervised Monocular Depth Estimation by Integrating IMU Motion Dynamicscited by 20
  3. 2022Transformers in Self-Supervised Monocular Depth Estimation with Unknown Camera Intrinsicscited by 19
  4. 2025Vision-Language Embodiment for Monocular Depth Estimationcited by 16
  5. 2023Learnable Differencing Center for Nighttime Depth Perceptioncited by 14
  6. 2024Embodiment: Self-Supervised Depth Estimation Based on Camera Modelscited by 9
  7. 2022Adversarial Attacks on Monocular Pose Estimationcited by 6
  8. 2022Image Masking for Robust Self-Supervised Monocular Depth Estimationcited by 5
  9. 2023Continual Learning of Unsupervised Monocular Depth from Videoscited by 5
  10. 2023Camera Height Doesn't Change: Unsupervised Training for Metric Monocular Road-Scene Depth Estimationcited by 3
  11. 2022AI-Driven Road Maintenance Inspection v2: Reducing Data Dependency & Quantifying Road Damagecited by 1
  12. 2023Do More With What You Have: Transferring Depth-Scale from Labeled to Unlabeled Domainscited by 1
4 more
  1. 2023Pose Constraints for Consistent Self-supervised Monocular Depth and Ego-motioncited by 0
  2. 2023Transformers in Unsupervised Structure-from-Motioncited by 0
  3. 2024Multiple Prior Representation Learning for Self-Supervised Monocular Depth Estimation via Hybrid Transformercited by 0
  4. 2025Depth3DLane: Fusing Monocular 3D Lane Detection with Self-Supervised Monocular Depth Estimationcited by 0