# Learning Depth from Monocular Videos using Direct Methods

arXiv: https://arxiv.org/abs/1712.00175 · HTML: https://mlsanity.com/paper/1712.00175/

## Key facts

- Cited by 154 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Dec 2017.

## Built on by (most cited first, 30 of 154)

- [BARF: Bundle-Adjusting Neural Radiance Fields](https://mlsanity.com/paper/2104.06405/) (2021)
- [SinNeRF: Training Neural Radiance Fields on Complex Scenes from a Single Image](https://mlsanity.com/paper/2204.00928/) (2022)
- [NeuralLift-360: Lifting An In-the-wild 2D Photo to A 3D Object with 360deg Views](https://mlsanity.com/paper/2211.16431/) (2022)
- [MonoViT: Self-Supervised Monocular Depth Estimation with a Vision Transformer](https://mlsanity.com/paper/2208.03543/) (2022)
- [MINE: Towards Continuous Depth MPI with NeRF for Novel View Synthesis](https://mlsanity.com/paper/2103.14910/) (2021)
- [Lite-Mono: A Lightweight CNN and Transformer Architecture for Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/2211.13202/) (2022)
- [Combining Events and Frames using Recurrent Asynchronous Multimodal Networks for Monocular Depth Prediction](https://mlsanity.com/paper/2102.09320/) (2021)
- [SurroundDepth: Entangling Surrounding Views for Self-Supervised Multi-Camera Depth Estimation](https://mlsanity.com/paper/2204.03636/) (2022)
- [Unsupervised Scale-consistent Depth Learning from Video](https://mlsanity.com/paper/2105.11610/) (2021)
- [Channel-Wise Attention-Based Network for Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/2112.13047/) (2021)
- [360MonoDepth: High-Resolution 360deg Monocular Depth Estimation](https://mlsanity.com/paper/2111.15669/) (2021)
- [Learning Monocular Depth in Dynamic Scenes via Instance-Aware Projection Consistency](https://mlsanity.com/paper/2102.02629/) (2021)
- [SC-DepthV3: Robust Self-supervised Monocular Depth Estimation for Dynamic Scenes](https://mlsanity.com/paper/2211.03660/) (2022)
- [Disentangling Object Motion and Occlusion for Unsupervised Multi-frame Monocular Depth](https://mlsanity.com/paper/2203.15174/) (2022)
- [Learning to Segment Rigid Motions from Two Frames](https://mlsanity.com/paper/2101.03694/) (2021)
- [Multi-View Depth Estimation by Fusing Single-View Depth Probability with Multi-View Geometry](https://mlsanity.com/paper/2112.08177/) (2021)
- [Self-supervised Monocular Depth Estimation for All Day Images using Domain Separation](https://mlsanity.com/paper/2108.07628/) (2021)
- [OccNeRF: Advancing 3D Occupancy Prediction in LiDAR-Free Environments](https://mlsanity.com/paper/2312.09243/) (2023)
- [R3D3: Dense 3D Reconstruction of Dynamic Scenes from Multiple Cameras](https://mlsanity.com/paper/2308.14713/) (2023)
- [MonoIndoor: Towards Good Practice of Self-Supervised Monocular Depth Estimation for Indoor Environments](https://mlsanity.com/paper/2107.12429/) (2021)
- [Single Image Depth Estimation: An Overview](https://mlsanity.com/paper/2104.06456/) (2021)
- [Diffusion Models for Monocular Depth Estimation: Overcoming Challenging Conditions](https://mlsanity.com/paper/2407.16698/) (2024)
- [Self-supervised Learning of Depth Inference for Multi-view Stereo](https://mlsanity.com/paper/2104.02972/) (2021)
- [Unsupervised Monocular Depth Estimation in Highly Complex Environments](https://mlsanity.com/paper/2107.13137/) (2021)
- [MGNet: Monocular Geometric Scene Understanding for Autonomous Driving](https://mlsanity.com/paper/2206.13199/) (2022)
- [Self-supervised Geometric Perception](https://mlsanity.com/paper/2103.03114/) (2021)
- [Attentive and Contrastive Learning for Joint Depth and Motion Field Estimation](https://mlsanity.com/paper/2110.06853/) (2021)
- [DevNet: Self-supervised Monocular Depth Learning via Density Volume Construction](https://mlsanity.com/paper/2209.06351/) (2022)
- [RM-Depth: Unsupervised Learning of Recurrent Monocular Depth in Dynamic Scenes](https://mlsanity.com/paper/2303.04456/) (2023)
- [Kick Back & Relax: Learning to Reconstruct the World by Watching SlowTV](https://mlsanity.com/paper/2307.10713/) (2023)

## Abstract (arXiv)

The ability to predict depth from a single image - using recent advances in CNNs - is of increasing interest to the vision community. Unsupervised strategies to learning are particularly appealing as they can utilize much larger and varied monocular video datasets during learning without the need for ground truth depth or stereo. In previous works, separate pose and depth CNN predictors had to be determined such that their joint outputs minimized the photometric error. Inspired by recent advances in direct visual odometry (DVO), we argue that the depth CNN predictor can be learned without a pose CNN predictor. Further, we demonstrate empirically that incorporation of a differentiable implementation of DVO, along with a novel depth normalization strategy - substantially improves performance over state of the art that use monocular videos for training.