Learning Depth from Monocular Videos using Direct Methods
Not on the map. Why?
Abstract (arXiv)
The ability to predict depth from a single image - using recent advances in CNNs - is of increasing interest to the vision community. Unsupervised strategies to learning are particularly appealing as they can utilize much larger and varied monocular video datasets during learning without the need for ground truth depth or stereo. In previous works, separate pose and depth CNN predictors had to be determined such that their joint outputs minimized the photometric error. Inspired by recent advances in direct visual odometry (DVO), we argue that the depth CNN predictor can be learned without a pose CNN predictor. Further, we demonstrate empirically that incorporation of a differentiable implementation of DVO, along with a novel depth normalization strategy - substantially improves performance over state of the art that use monocular videos for training.
Key facts. Cited by 154 papers in the map (data through 14 Oct 2026). First posted on arXiv 1 Dec 2017.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 49; 2022: 41; 2023: 29; 2024: 17; 2025: 5; 2026: 13.
154 papers.
Sort:
By area:
- 2021BARF: Bundle-Adjusting Neural Radiance Fieldscited by 642
- 2022SinNeRF: Training Neural Radiance Fields on Complex Scenes from a Single Imagecited by 165
- 2022NeuralLift-360: Lifting An In-the-wild 2D Photo to A 3D Object with 360deg Viewscited by 148
- 2022MonoViT: Self-Supervised Monocular Depth Estimation with a Vision Transformercited by 136
- 2021MINE: Towards Continuous Depth MPI with NeRF for Novel View Synthesiscited by 133
- 2022Lite-Mono: A Lightweight CNN and Transformer Architecture for Self-Supervised Monocular Depth Estimationcited by 100
- 2021Combining Events and Frames using Recurrent Asynchronous Multimodal Networks for Monocular Depth Predictioncited by 90
- 2022SurroundDepth: Entangling Surrounding Views for Self-Supervised Multi-Camera Depth Estimationcited by 76
- 2021Unsupervised Scale-consistent Depth Learning from Videocited by 69
- 2021Channel-Wise Attention-Based Network for Self-Supervised Monocular Depth Estimationcited by 63
- 2021360MonoDepth: High-Resolution 360deg Monocular Depth Estimationcited by 60
- 2021Learning Monocular Depth in Dynamic Scenes via Instance-Aware Projection Consistencycited by 58
38 more
- 2022SC-DepthV3: Robust Self-supervised Monocular Depth Estimation for Dynamic Scenescited by 52
- 2022Disentangling Object Motion and Occlusion for Unsupervised Multi-frame Monocular Depthcited by 48
- 2021Learning to Segment Rigid Motions from Two Framescited by 46
- 2021Multi-View Depth Estimation by Fusing Single-View Depth Probability with Multi-View Geometrycited by 46
- 2021Self-supervised Monocular Depth Estimation for All Day Images using Domain Separationcited by 44
- 2023OccNeRF: Advancing 3D Occupancy Prediction in LiDAR-Free Environmentscited by 41
- 2023R3D3: Dense 3D Reconstruction of Dynamic Scenes from Multiple Camerascited by 39
- 2021MonoIndoor: Towards Good Practice of Self-Supervised Monocular Depth Estimation for Indoor Environmentscited by 38
- 2021Single Image Depth Estimation: An Overviewcited by 35
- 2024Diffusion Models for Monocular Depth Estimation: Overcoming Challenging Conditionscited by 29
- 2021Self-supervised Learning of Depth Inference for Multi-view Stereocited by 28
- 2021Unsupervised Monocular Depth Estimation in Highly Complex Environmentscited by 28
- 2022MGNet: Monocular Geometric Scene Understanding for Autonomous Drivingcited by 27
- 2021Self-supervised Geometric Perceptioncited by 26
- 2021Attentive and Contrastive Learning for Joint Depth and Motion Field Estimationcited by 24
- 2022DevNet: Self-supervised Monocular Depth Learning via Density Volume Constructioncited by 22
- 2023RM-Depth: Unsupervised Learning of Recurrent Monocular Depth in Dynamic Scenescited by 22
- 2023Kick Back & Relax: Learning to Reconstruct the World by Watching SlowTVcited by 21
- 2023SQLdepth: Generalizable Self-Supervised Fine-Structured Monocular Depth Estimationcited by 21
- 2023Dynamo-Depth: Fixing Unsupervised Depth Estimation for Dynamical Scenescited by 20
- 2021Learning Depth via Leveraging Semantics: Self-supervised Monocular Depth Estimation with Both Implicit and Explicit Semantic Guidancecited by 19
- 2021Outdoor inverse rendering from a single image using multiview self-supervisioncited by 18
- 2022Deconstructing Self-Supervised Monocular Reconstruction: The Design Decisions that Mattercited by 18
- 2021VOLDOR: Visual Odometry from Log-logistic Dense Optical flow Residualscited by 17
- 2023Adversarial Training of Self-supervised Monocular Depth Estimation against Physical-World Attackscited by 17
- 2026FD-VLA: Force-Distilled Vision-Language-Action Model for Contact-Rich Manipulationcited by 17
- 2022Multi-Camera Collaborative Depth Prediction via Consistent Structure Estimationcited by 16
- 2023DualRefine: Self-Supervised Depth and Pose Estimation Through Iterative Epipolar Sampling and Refinement Toward Equilibriumcited by 16
- 2023Multi-Object Discovery by Low-Dimensional Object Motioncited by 16
- 2023AugUndo: Scaling Up Augmentations for Monocular Depth Completion and Estimationcited by 15
- 2022PanopticDepth: A Unified Framework for Depth-aware Panoptic Segmentationcited by 14
- 2021Advancing Self-supervised Monocular Depth Learning with Sparse LiDARcited by 13
- 2022The Monocular Depth Estimation Challengecited by 13
- 2023The RoboDepth Challenge: Methods and Advancements Towards Robust Depth Estimationcited by 13
- 2023MonoDiffusion: Self-Supervised Monocular Depth Estimation Using Diffusion Modelcited by 13
- 2021H-Net: Unsupervised Attention-based Stereo Depth Estimation Leveraging Epipolar Geometrycited by 12
- 2021RGB-D Salient Object Detection with Ubiquitous Target Awarenesscited by 12
- 2021PLNet: Plane and Line Priors for Unsupervised Indoor Depth Estimationcited by 12
50 most cited of 154 All 154 (JSON)
- 2026XiDepth: a Lightweight and Efficient Network for Self-supervised Monocular Depth Estimation4 Aug 2026
- 2026JEPADepth: Masked Predictive Representation Learning for Self-Supervised Monocular Depth Estimation29 Jul 2026
- 2026Towards Robust Driving Perception: A Flexible Scale-Driven Family for Self-Supervised Monocular Depth Estimation1 Jul 2026
- 2026VolFill: Single-View Amodal 3D Scene Reconstruction with Volumetric Flow Matching29 May 2026
- 2026SA4Depth: Consistent Pose-Depth Scale Alignment for Self-Supervised Monocular Depth Estimation27 May 2026
- 2026Improved monocular depth prediction using distance transform over pre-semantic contours with self-supervised neural networks8 May 2026
- 2026Rebalancing gradient to improve self-supervised co-training of depth, odometry and optical flow predictions8 May 2026
- 2026LiftFormer: Lifting and Frame Theory Based Monocular Depth Estimation Using Depth and Edge Oriented Subspace Representation8 Apr 2026
- 2026DAGE: Dual-Stream Architecture for Efficient and Fine-Grained Geometry Estimation4 Mar 2026
- 2026Depth from Defocus via Direct Optimization18 Feb 2026
- 2026NeRFscopy: Neural Radiance Fields for in-vivo Time-Varying Tissues from Endoscopy17 Feb 2026
- 2026PuriLight: A Lightweight Shuffle and Purification Framework for Monocular Depth Estimation11 Feb 2026
38 more
- 2026FD-VLA: Force-Distilled Vision-Language-Action Model for Contact-Rich Manipulation2 Feb 2026
- 2025Uncertainty-aware Test-Time Training (UT3) for Efficient On-the-fly Domain Adaptive Dense Regression3 Sep 2025
- 2025Extending Foundational Monocular Depth Estimators to Fisheye Cameras with Calibration Tokens6 Aug 2025
- 2025The Fourth Monocular Depth Estimation Challenge24 Apr 2025
- 2025Endo-FASt3r: Endoscopic Foundation model Adaptation for Structure from motion10 Mar 2025
- 2025Mean of Means: Human Localization with Calibration-free and Unconstrained Camera Settings (extended version)18 Feb 2025
- 2024On-Device Self-Supervised Learning of Low-Latency Monocular Depth from Only Events9 Dec 2024
- 2024Self-supervised Monocular Depth and Pose Estimation for Endoscopy with Latent Priors26 Nov 2024
- 2024PMPNet: Pixel Movement Prediction Network for Monocular Depth Estimation in Dynamic Scenes4 Nov 2024
- 2024Optical Lens Attack on Monocular Depth Estimation for Autonomous Driving31 Oct 2024
- 2024Pseudo-Stereo Inputs: A Solution to the Occlusion Challenge in Self-Supervised Stereo Matching3 Oct 2024
- 2024Optical Lens Attack on Deep Learning Based Monocular Depth Estimation25 Sep 2024
- 2024Embodiment: Self-Supervised Depth Estimation Based on Camera Models2 Aug 2024
- 2024Mean of Means: A 10-dollar Solution for Human Localization with Calibration-free and Unconstrained Camera Settings30 Jul 2024
- 2024Diffusion Models for Monocular Depth Estimation: Overcoming Challenging Conditions23 Jul 2024
- 2024Mono-ViFI: A Unified Learning Framework for Self-supervised Single- and Multi-frame Monocular Depth Estimation19 Jul 2024
- 2024Self-supervised Pretraining and Finetuning for Monocular Depth and Visual Odometry16 Jun 2024
- 2024Self-supervised Adversarial Training of Monocular Depth Estimation against Physical-World Attacks9 Jun 2024
- 2024MambaDepth: Enhancing Long-range Dependency for Self-Supervised Fine-Structured Monocular Depth Estimation6 Jun 2024
- 2024MCDS-VSS: Moving Camera Dynamic Scene Video Semantic Segmentation by Filtering with Self-Supervised Geometry and Motion30 May 2024
- 2024SPIdepth: Strengthened Pose Information for Self-supervised Monocular Depth Estimation18 Apr 2024
- 2024Adaptive Discrete Disparity Volume for Self-supervised Monocular Depth Estimation4 Apr 2024
- 2024Kick Back & Relax++: Scaling Beyond Ground-Truth Depth with SlowTV & CribsTV3 Mar 2024
- 2023Manydepth2: Motion-Aware Self-Supervised Monocular Depth Estimation in Dynamic Scenes23 Dec 2023
- 2023OccNeRF: Advancing 3D Occupancy Prediction in LiDAR-Free Environments14 Dec 2023
- 2023Camera Height Doesn't Change: Unsupervised Training for Metric Monocular Road-Scene Depth Estimation7 Dec 2023
- 2023MonoDiffusion: Self-Supervised Monocular Depth Estimation Using Diffusion Model13 Nov 2023
- 2023Continual Learning of Unsupervised Monocular Depth from Videos4 Nov 2023
- 2023Dynamo-Depth: Fixing Unsupervised Depth Estimation for Dynamical Scenes29 Oct 2023
- 2023Learning depth from monocular video sequences26 Oct 2023
- 2023AugUndo: Scaling Up Augmentations for Monocular Depth Completion and Estimation15 Oct 2023
- 2023Towards Better Data Exploitation in Self-Supervised Monocular Depth Estimation11 Sep 2023
- 2023SQLdepth: Generalizable Self-Supervised Fine-Structured Monocular Depth Estimation1 Sep 2023
- 2023R3D3: Dense 3D Reconstruction of Dynamic Scenes from Multiple Cameras28 Aug 2023
- 2023DS-Depth: Dynamic and Static Depth Estimation via a Fusion Cost Volume14 Aug 2023
- 2023EndoDepthL: Lightweight Endoscopic Monocular Depth Estimation with CNN-Transformer4 Aug 2023
- 2023The RoboDepth Challenge: Methods and Advancements Towards Robust Depth Estimation27 Jul 2023
- 2023LiDAR Meta Depth Completion24 Jul 2023
50 newest of 154 All 154 (JSON)
- 2022NeuralLift-360: Lifting An In-the-wild 2D Photo to A 3D Object with 360deg Viewscited by 148
- 2024Diffusion Models for Monocular Depth Estimation: Overcoming Challenging Conditionscited by 29
- 2023MonoDiffusion: Self-Supervised Monocular Depth Estimation Using Diffusion Modelcited by 13
- 2026VolFill: Single-View Amodal 3D Scene Reconstruction with Volumetric Flow Matchingcited by 0
- 2026VolFill: Single-View Amodal 3D Scene Reconstruction with Volumetric Flow Matching29 May 2026
- 2024Diffusion Models for Monocular Depth Estimation: Overcoming Challenging Conditions23 Jul 2024
- 2023MonoDiffusion: Self-Supervised Monocular Depth Estimation Using Diffusion Model13 Nov 2023
- 2022NeuralLift-360: Lifting An In-the-wild 2D Photo to A 3D Object with 360deg Views29 Nov 2022
- 2021BARF: Bundle-Adjusting Neural Radiance Fieldscited by 642
- 2022SinNeRF: Training Neural Radiance Fields on Complex Scenes from a Single Imagecited by 165
- 2022MonoViT: Self-Supervised Monocular Depth Estimation with a Vision Transformercited by 136
- 2021MINE: Towards Continuous Depth MPI with NeRF for Novel View Synthesiscited by 133
- 2022Lite-Mono: A Lightweight CNN and Transformer Architecture for Self-Supervised Monocular Depth Estimationcited by 100
- 2021Combining Events and Frames using Recurrent Asynchronous Multimodal Networks for Monocular Depth Predictioncited by 90
- 2022SurroundDepth: Entangling Surrounding Views for Self-Supervised Multi-Camera Depth Estimationcited by 76
- 2021Unsupervised Scale-consistent Depth Learning from Videocited by 69
- 2021Channel-Wise Attention-Based Network for Self-Supervised Monocular Depth Estimationcited by 63
- 2021360MonoDepth: High-Resolution 360deg Monocular Depth Estimationcited by 60
- 2021Learning Monocular Depth in Dynamic Scenes via Instance-Aware Projection Consistencycited by 58
- 2022SC-DepthV3: Robust Self-supervised Monocular Depth Estimation for Dynamic Scenescited by 52
12 most cited of 146 All 146 (JSON)
- 2026XiDepth: a Lightweight and Efficient Network for Self-supervised Monocular Depth Estimation4 Aug 2026
- 2026JEPADepth: Masked Predictive Representation Learning for Self-Supervised Monocular Depth Estimation29 Jul 2026
- 2026Towards Robust Driving Perception: A Flexible Scale-Driven Family for Self-Supervised Monocular Depth Estimation1 Jul 2026
- 2026SA4Depth: Consistent Pose-Depth Scale Alignment for Self-Supervised Monocular Depth Estimation27 May 2026
- 2026Improved monocular depth prediction using distance transform over pre-semantic contours with self-supervised neural networks8 May 2026
- 2026Rebalancing gradient to improve self-supervised co-training of depth, odometry and optical flow predictions8 May 2026
- 2026LiftFormer: Lifting and Frame Theory Based Monocular Depth Estimation Using Depth and Edge Oriented Subspace Representation8 Apr 2026
- 2026DAGE: Dual-Stream Architecture for Efficient and Fine-Grained Geometry Estimation4 Mar 2026
- 2026Depth from Defocus via Direct Optimization18 Feb 2026
- 2026NeRFscopy: Neural Radiance Fields for in-vivo Time-Varying Tissues from Endoscopy17 Feb 2026
- 2026PuriLight: A Lightweight Shuffle and Purification Framework for Monocular Depth Estimation11 Feb 2026
- 2025Uncertainty-aware Test-Time Training (UT3) for Efficient On-the-fly Domain Adaptive Dense Regression3 Sep 2025
12 newest of 146 All 146 (JSON)