# Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Video

arXiv: https://arxiv.org/abs/1908.10553 · HTML: https://mlsanity.com/paper/1908.10553/

## Key facts

- Cited by 244 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Aug 2019.

## Built on by (most cited first, 30 of 244)

- [SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Driving](https://mlsanity.com/paper/2303.09551/) (2023)
- [The Temporal Opportunist: Self-Supervised Multi-Frame Monocular Depth](https://mlsanity.com/paper/2104.14540/) (2021)
- [Transformer-Based Attention Networks for Continuous Pixel-Wise Prediction](https://mlsanity.com/paper/2103.12091/) (2021)
- [SurroundDepth: Entangling Surrounding Views for Self-Supervised Multi-Camera Depth Estimation](https://mlsanity.com/paper/2204.03636/) (2022)
- [Self-Supervised Monocular Depth and Ego-Motion Estimation in Endoscopy: Appearance Flow to the Rescue](https://mlsanity.com/paper/2112.08122/) (2021)
- [Unsupervised Scale-consistent Depth Learning from Video](https://mlsanity.com/paper/2105.11610/) (2021)
- [Learning Monocular Depth in Dynamic Scenes via Instance-Aware Projection Consistency](https://mlsanity.com/paper/2102.02629/) (2021)
- [Regularizing Nighttime Weirdness: Efficient Self-supervised Monocular Depth Estimation in the Dark](https://mlsanity.com/paper/2108.03830/) (2021)
- [Fine-grained Semantics-aware Representation Enhancement for Self-supervised Monocular Depth Estimation](https://mlsanity.com/paper/2108.08829/) (2021)
- [Hierarchical Normalization for Robust Monocular Depth Estimation](https://mlsanity.com/paper/2210.09670/) (2022)
- [Surgical-DINO: Adapter Learning of Foundation Models for Depth Estimation in Endoscopic Surgery](https://mlsanity.com/paper/2401.06013/) (2024)
- [Virtual Normal: Enforcing Geometric Constraints for Accurate and Robust Depth Prediction](https://mlsanity.com/paper/2103.04216/) (2021)
- [EndoDAC: Efficient Adapting Foundation Model for Self-Supervised Depth Estimation from Any Endoscopic Camera](https://mlsanity.com/paper/2405.08672/) (2024)
- [Towards Accurate Reconstruction of 3D Scene Shape from A Single Monocular Image](https://mlsanity.com/paper/2208.13241/) (2022)
- [SelfSplat: Pose-Free and 3D Prior-Free Generalizable 3D Gaussian Splatting](https://mlsanity.com/paper/2411.17190/) (2024)
- [Full Surround Monodepth from Multiple Cameras](https://mlsanity.com/paper/2104.00152/) (2021)
- [OccNeRF: Advancing 3D Occupancy Prediction in LiDAR-Free Environments](https://mlsanity.com/paper/2312.09243/) (2023)
- [Sparse Auxiliary Networks for Unified Monocular Depth Prediction and Completion](https://mlsanity.com/paper/2103.16690/) (2021)
- [MonoIndoor: Towards Good Practice of Self-Supervised Monocular Depth Estimation for Indoor Environments](https://mlsanity.com/paper/2107.12429/) (2021)
- [RA-Depth: Resolution Adaptive Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/2207.11984/) (2022)
- [Single Image Depth Prediction with Wavelet Decomposition](https://mlsanity.com/paper/2106.02022/) (2021)
- [Diffusion Models for Monocular Depth Estimation: Overcoming Challenging Conditions](https://mlsanity.com/paper/2407.16698/) (2024)
- [Driv3R: Learning Dense 4D Reconstruction for Autonomous Driving](https://mlsanity.com/paper/2412.06777/) (2024)
- [Unsupervised Monocular Depth Estimation in Highly Complex Environments](https://mlsanity.com/paper/2107.13137/) (2021)
- [NARUTO: Neural Active Reconstruction from Uncertain Target Observations](https://mlsanity.com/paper/2402.18771/) (2024)
- [Attention meets Geometry: Geometry Guided Spatial-Temporal Attention for Consistent Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/2110.08192/) (2021)
- [DesNet: Decomposed Scale-Consistent Network for Unsupervised Depth Completion](https://mlsanity.com/paper/2211.10994/) (2022)
- [Gradient-based Uncertainty for Monocular Depth Estimation](https://mlsanity.com/paper/2208.02005/) (2022)
- [Attentive and Contrastive Learning for Joint Depth and Motion Field Estimation](https://mlsanity.com/paper/2110.06853/) (2021)
- [Multi-Modal Masked Pre-Training for Monocular Panoramic Depth Completion](https://mlsanity.com/paper/2203.09855/) (2022)

## Abstract (arXiv)

Recent work has shown that CNN-based depth and ego-motion estimators can be learned using unlabelled monocular videos. However, the performance is limited by unidentified moving objects that violate the underlying static scene assumption in geometric image reconstruction. More significantly, due to lack of proper constraints, networks output scale-inconsistent results over different samples, i.e., the ego-motion network cannot provide full camera trajectories over a long video sequence because of the per-frame scale ambiguity. This paper tackles these challenges by proposing a geometry consistency loss for scale-consistent predictions and an induced self-discovered mask for handling moving objects and occlusions. Since we do not leverage multi-task learning like recent works, our framework is much simpler and more efficient. Comprehensive evaluation results demonstrate that our depth estimator achieves the state-of-the-art performance on the KITTI dataset. Moreover, we show that our ego-motion network is able to predict a globally scale-consistent camera trajectory for long video sequences, and the resulting visual odometry accuracy is competitive with the recent model that is trained using stereo videos. To the best of our knowledge, this is the first work to show that deep networks trained using unlabelled monocular videos can predict globally scale-consistent camera trajectories over a long video sequence.