mlsanityData to 14 Oct 2026

Unsupervised Scale-consistent Depth Learning from Video

arXiv 2105.11610 ↗First posted 25 May 2021cs.CVcited by 69

Not on the map. Why?

Abstract (arXiv)

We propose a monocular depth estimator SC-Depth, which requires only unlabelled videos for training and enables the scale-consistent prediction at inference time. Our contributions include: (i) we propose a geometry consistency loss, which penalizes the inconsistency of predicted depths between adjacent views; (ii) we propose a self-discovered mask to automatically localize moving objects that violate the underlying static scene assumption and cause noisy signals during training; (iii) we demonstrate the efficacy of each component with a detailed ablation study and show high-quality depth estimation results in both KITTI and NYUv2 datasets. Moreover, thanks to the capability of scale-consistent prediction, we show that our monocular-trained deep networks are readily integrated into the ORB-SLAM2 system for more robust and accurate tracking. The proposed hybrid Pseudo-RGBD SLAM shows compelling results in KITTI, and it generalizes well to the KAIST dataset without additional training. Finally, we provide several demos for qualitative evaluation.

Key facts. Cited by 69 papers in the map (data through 14 Oct 2026). Builds on 65 of its 99 references that are in the map. First posted on arXiv 25 May 2021.

Builds on

65 of 99 references.

References · 65

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  4. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,495
  5. 2015Spatial Transformer Networkscited by 1,429
  6. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,274
  7. 2015Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)cited by 1,070
  8. 2015ORB-SLAM: a Versatile and Accurate Monocular SLAM Systemcited by 988
  9. 2016Unsupervised Monocular Depth Estimation with Left-Right Consistencycited by 820
  10. 2016ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Camerascited by 806
  11. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 782
  12. 2018MegaDepth: Learning Single-View Depth Prediction from Internet Photoscited by 737
38 more
  1. 2018Deep Ordinal Regression Network for Monocular Depth Estimationcited by 723
  2. 2018Open3D: A Modern Library for 3D Data Processingcited by 721
  3. 2016Direct Sparse Odometrycited by 485
  4. 2014Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecturecited by 483
  5. 20193D Packing for Self-Supervised Monocular Depth Estimationcited by 439
  6. 2016Deeper Depth Prediction with Fully Convolutional Residual Networkscited by 403
  7. 2016Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescuecited by 368
  8. 2018GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Posecited by 342
  9. 2019Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Videocited by 244
  10. 2019Enforcing geometric constraints of virtual normal for depth predictioncited by 234
  11. 2018Competitive Collaboration: Joint Unsupervised Learning of Depth, Camera Motion, Optical Flow and Motion Segmentationcited by 222
  12. 2020Consistent Video Depth Estimationcited by 220
  13. 2020Learning to Recover 3D Scene Shape from a Single Imagecited by 216
  14. 2020D3VO: Deep Depth, Deep Pose and Deep Uncertainty for Monocular Visual Odometrycited by 206
  15. 2020On the uncertainty of self-supervised monocular depth estimationcited by 182
  16. 2018Unsupervised Learning of Depth and Ego-Motion from Monocular Video Using 3D Geometric Constraintscited by 179
  17. 2019Depth from Videos in the Wild: Unsupervised Monocular Depth Learning from Unknown Camerascited by 170
  18. 2015Learning Depth from Single Monocular Images Using Deep Convolutional Neural Fieldscited by 169
  19. 2018Unsupervised Learning of Monocular Depth Estimation and Visual Odometry with Deep Feature Reconstructioncited by 159
  20. 2018Depth Prediction Without the Sensors: Leveraging Structure for Unsupervised Learning from Monocular Videoscited by 155
  21. 2017Learning Depth from Monocular Videos using Direct Methodscited by 154
  22. 2017CNN-SLAM: Real-time dense monocular SLAM with learned depth predictioncited by 152
  23. 2018CodeSLAM - Learning a Compact, Optimisable Representation for Dense Visual SLAMcited by 152
  24. 2018DF-Net: Unsupervised Joint Learning of Depth and Flow using Cross-Task Consistencycited by 148
  25. 2020Self-Supervised Monocular Depth Estimation: Solving the Dynamic Object Problem by Semantic Guidancecited by 148
  26. 2020Semantically-Guided Representation Learning for Self-Supervised Monocular Depthcited by 134
  27. 2019Self-supervised Learning with Geometric Constraints in Monocular Video: Connecting Flow, Depth, and Cameracited by 130
  28. 2019Learning the Depths of Moving People by Watching Frozen Peoplecited by 120
  29. 2017Semi-Supervised Deep Learning for Monocular Depth Map Predictioncited by 109
  30. 2018Every Pixel Counts ++: Joint Learning of Geometry and Motion with 3D Holistic Understandingcited by 101
  31. 2017UnDeepVO: Monocular Visual Odometry through Unsupervised Deep Learningcited by 95
  32. 2017SfM-Net: Learning of Structure and Motion from Videocited by 89
  33. 2018Deep Virtual Stereo Odometry: Leveraging Deep Depth Prediction for Monocular Direct Sparse Odometrycited by 85
  34. 2020Towards Better Generalization: Joint Depth-Pose Learning without PoseNetcited by 85
  35. 2020Unsupervised Monocular Depth Learning in Dynamic Scenescited by 84
  36. 2020Guiding Monocular Depth Estimation Using Depth-Attention Volumecited by 82
  37. 2019Web Stereo Video Supervision for Depth Prediction from Dynamic Scenescited by 73
  38. 2018SuperDepth: Self-Supervised, Super-Resolved Monocular Depth Estimationcited by 64

50 most cited of 65 All 65 (JSON)

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 14; 2023: 22; 2024: 16; 2025: 15.

69 papers.

Sort:

By area:

  1. 2022SC-DepthV3: Robust Self-supervised Monocular Depth Estimation for Dynamic Scenescited by 52
  2. 2021Toward Practical Monocular Indoor Depth Estimationcited by 42
  3. 2022BiFuse++: Self-supervised and Efficient Bi-projection Fusion for 360 Depth Estimationcited by 35
  4. 2023Self-supervised Monocular Depth Estimation: Let's Talk About The Weathercited by 34
  5. 2024WorDepth: Variational Language Prior for Monocular Depth Estimationcited by 34
  6. 2025Synthetic-to-Real Self-supervised Robust Depth Estimation via Learning with Motion and Structure Priorscited by 22
  7. 2024RSA: Resolving Scale Ambiguities in Monocular Depth Estimators through Language Descriptionscited by 21
  8. 2023SimCol3D -- 3D Reconstruction during Colonoscopy Challengecited by 19
  9. 2023DualRefine: Self-Supervised Depth and Pose Estimation Through Iterative Epipolar Sampling and Refinement Toward Equilibriumcited by 16
  10. 2024Incremental Joint Learning of Depth, Pose and Implicit Scene Representation on Monocular Camera in Large-scale Scenescited by 15
  11. 2022Maximizing Self-supervision from Thermal Image for Effective Self-supervised Learning of Depth and Ego-motioncited by 14
  12. 2023GasMono: Geometry-Aided Self-Supervised Monocular Depth Estimation for Indoor Scenescited by 13
38 more
  1. 2024DCPI-Depth: Explicitly Infusing Dense Correspondence Prior to Unsupervised Monocular Depth Estimationcited by 13
  2. 2023Manydepth2: Motion-Aware Self-Supervised Monocular Depth Estimation in Dynamic Scenescited by 12
  3. 2022Towards 3D Scene Reconstruction from Locally Scale-Aligned Monocular Video Depthcited by 11
  4. 2023WS-SfMLearner: Self-supervised Monocular Depth and Ego-motion Estimation on Surgical Videos with Unknown Camera Parameterscited by 9
  5. 2023ADFactory: An Effective Framework for Generalizing Optical Flow with Nerfcited by 9
  6. 2022Depth-CUPRL: Depth-Imaged Contrastive Unsupervised Prioritized Representations in Reinforcement Learning for Mapless Navigation of Unmanned Aerial Vehiclescited by 8
  7. 2023Scalable Semantic 3D Mapping of Coral Reefs with Deep Learningcited by 8
  8. 2024Indoor Obstacle Discovery on Reflective Ground via Monocular Cameracited by 8
  9. 2022SphereDepth: Panorama Depth Estimation from Spherical Domaincited by 7
  10. 2023Deeper into Self-Supervised Monocular Indoor Depth Estimationcited by 7
  11. 2024Out of the Room: Generalizing Event-Based Dynamic Motion Segmentation for Complex Scenescited by 7
  12. 2024SCIPaD: Incorporating Spatial Clues into Unsupervised Pose-Depth Joint Learningcited by 7
  13. 2021MotionHint: Self-Supervised Monocular Visual Odometry with Motion Constraintscited by 6
  14. 2022Adversarial Attacks on Monocular Pose Estimationcited by 6
  15. 2022Towards Domain-agnostic Depth Completioncited by 6
  16. 2023NVDS+: Towards Efficient and Versatile Neural Stabilizer for Video Depth Estimationcited by 6
  17. 2025SAB3R: Semantic-Augmented Backbone in 3D Reconstructioncited by 6
  18. 2025Hybrid-grained Feature Aggregation with Coarse-to-fine Language Guidance for Self-supervised Monocular Depth Estimationcited by 6
  19. 2022SelfTune: Metrically Scaled Monocular Depth Estimation through Self-Supervised Learningcited by 4
  20. 2023TransFusionOdom: Interpretable Transformer-based LiDAR-Inertial Fusion Odometry Estimationcited by 4
  21. 2023MeSa: Masked, Geometric, and Supervised Pre-training for Monocular Depth Estimationcited by 4
  22. 2024TanDepth: Leveraging Global DEMs for Metric Monocular Depth Estimation in UAVscited by 4
  23. 2025PosePilot: Steering Camera Pose for Generative World Models with Self-supervised Depthcited by 4
  24. 2025ER-LoRA: Effective-Rank Guided Adaptation for Weather-Generalized Depth Estimationcited by 4
  25. 2022DeepMLE: A Robust Deep Maximum Likelihood Estimator for Two-view Structure from Motioncited by 3
  26. 2025Bridging Spectral-wise and Multi-spectral Depth Estimation via Geometry-guided Contrastive Learningcited by 3
  27. 2022RAUM-VO: Rotational Adjusted Unsupervised Monocular Visual Odometrycited by 2
  28. 2022CbwLoss: Constrained Bidirectional Weighted Loss for Self-supervised Learning of Depth and Posecited by 2
  29. 2024ToDER: Towards Colonoscopy Depth Estimation and Reconstruction with Geometry Constraint Adaptationcited by 2
  30. 2024Structure-Centric Robust Monocular Depth Estimation via Knowledge Distillationcited by 2
  31. 2025SphereFusion: Efficient Panorama Depth Estimation via Gated Fusioncited by 2
  32. 2022Self-Supervised Depth Estimation with Isometric-Self-Sample-Based Learningcited by 1
  33. 2023Depth- and Semantics-aware Multi-modal Domain Translation: Generating 3D Panoramic Color Images from LiDAR Point Cloudscited by 1
  34. 2023HQDec: Self-Supervised Monocular Depth Estimation Based on a High-Quality Decodercited by 1
  35. 2023RGB-D-Fusion: Image Conditioned Depth Diffusion of Humanoid Subjectscited by 1
  36. 2023Unleashing the Power of Depth and Pose Estimation Neural Networks by Designing Compatible Endoscopic Imagescited by 1
  37. 2023Improving Neural Indoor Surface Reconstruction with Mask-Guided Adaptive Consistency Constraintscited by 1
  38. 2023InSpaceType: Reconsider Space Type in Indoor Monocular Depth Estimationcited by 1

50 most cited of 69 All 69 (JSON)