# D3VO: Deep Depth, Deep Pose and Deep Uncertainty for Monocular Visual Odometry

arXiv: https://arxiv.org/abs/2003.01060 · HTML: https://mlsanity.com/paper/2003.01060/

## Key facts

- Cited by 206 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Mar 2020.

## Built on by (most cited first, 30 of 206)

- [DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Cameras](https://mlsanity.com/paper/2108.10869/) (2021)
- [A Survey of Uncertainty in Deep Neural Networks](https://mlsanity.com/paper/2107.03342/) (2021)
- [NICE-SLAM: Neural Implicit Scalable Encoding for SLAM](https://mlsanity.com/paper/2112.12130/) (2021)
- [Continuous 3D Perception Model with Persistent State](https://mlsanity.com/paper/2501.12387/) (2025)
- [ConceptFusion: Open-set Multimodal 3D Mapping](https://mlsanity.com/paper/2302.07241/) (2023)
- [MegaSaM: Accurate, Fast, and Robust Structure and Motion from Casual Dynamic Videos](https://mlsanity.com/paper/2412.04463/) (2024)
- [Co-SLAM: Joint Coordinate and Sparse Parametric Encodings for Neural Real-Time SLAM](https://mlsanity.com/paper/2304.14377/) (2023)
- [Photo-SLAM: Real-time Simultaneous Localization and Photorealistic Mapping for Monocular, Stereo, and RGB-D Cameras](https://mlsanity.com/paper/2311.16728/) (2023)
- [MonoViT: Self-Supervised Monocular Depth Estimation with a Vision Transformer](https://mlsanity.com/paper/2208.03543/) (2022)
- [RGBD Objects in the Wild: Scaling Real-World 3D Object Learning from RGB-D Videos](https://mlsanity.com/paper/2401.12592/) (2024)
- [Boosting Monocular Depth Estimation Models to High-Resolution via Content-Adaptive Multi-Resolution Merging](https://mlsanity.com/paper/2105.14021/) (2021)
- [Lite-Mono: A Lightweight CNN and Transformer Architecture for Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/2211.13202/) (2022)
- [RelPose++: Recovering 6D Poses from Sparse-view Observations](https://mlsanity.com/paper/2305.04926/) (2023)
- [NeRF On-the-go: Exploiting Uncertainty for Distractor-free NeRFs in the Wild](https://mlsanity.com/paper/2405.18715/) (2024)
- [Self-Supervised Monocular Depth and Ego-Motion Estimation in Endoscopy: Appearance Flow to the Rescue](https://mlsanity.com/paper/2112.08122/) (2021)
- [Deep Patch Visual SLAM](https://mlsanity.com/paper/2408.01654/) (2024)
- [Unsupervised Scale-consistent Depth Learning from Video](https://mlsanity.com/paper/2105.11610/) (2021)
- [CG-SLAM: Efficient Dense RGB-D SLAM in a Consistent Uncertainty-aware 3D Gaussian Field](https://mlsanity.com/paper/2403.16095/) (2024)
- [Physical Attack on Monocular Depth Estimation with Optimal Adversarial Patches](https://mlsanity.com/paper/2207.04718/) (2022)
- [LEAP-VO: Long-term Effective Any Point Tracking for Visual Odometry](https://mlsanity.com/paper/2401.01887/) (2024)
- [VA-DepthNet: A Variational Approach to Single Image Depth Prediction](https://mlsanity.com/paper/2302.06556/) (2023)
- [SparsePose: Sparse-View Camera Pose Regression and Refinement](https://mlsanity.com/paper/2211.16991/) (2022)
- [Wide-Baseline Relative Camera Pose Estimation with Directional Learning](https://mlsanity.com/paper/2106.03336/) (2021)
- [EgoLocate: Real-time Motion Capture, Localization, and Mapping with Sparse Body-mounted Sensors](https://mlsanity.com/paper/2305.01599/) (2023)
- [PhyRecon: Physically Plausible Neural Scene Reconstruction](https://mlsanity.com/paper/2404.16666/) (2024)
- [LONG3R: Long Sequence Streaming 3D Reconstruction](https://mlsanity.com/paper/2507.18255/) (2025)
- [Toward Practical Monocular Indoor Depth Estimation](https://mlsanity.com/paper/2112.02306/) (2021)
- [Multimodal Fusion and Vision-Language Models: A Survey for Robot Vision](https://mlsanity.com/paper/2504.02477/) (2025)
- [Sparse Auxiliary Networks for Unified Monocular Depth Prediction and Completion](https://mlsanity.com/paper/2103.16690/) (2021)
- [TANDEM: Tracking and Dense Mapping in Real-time using Deep Multi-view Stereo](https://mlsanity.com/paper/2111.07418/) (2021)

## Abstract (arXiv)

We propose D3VO as a novel framework for monocular visual odometry that exploits deep networks on three levels -- deep depth, pose and uncertainty estimation. We first propose a novel self-supervised monocular depth estimation network trained on stereo videos without any external supervision. In particular, it aligns the training image pairs into similar lighting condition with predictive brightness transformation parameters. Besides, we model the photometric uncertainties of pixels on the input images, which improves the depth estimation accuracy and provides a learned weighting function for the photometric residuals in direct (feature-less) visual odometry. Evaluation results show that the proposed network outperforms state-of-the-art self-supervised depth estimation networks. D3VO tightly incorporates the predicted depth, pose and uncertainty into a direct visual odometry method to boost both the front-end tracking as well as the back-end non-linear optimization. We evaluate D3VO in terms of monocular visual odometry on both the KITTI odometry benchmark and the EuRoC MAV dataset.The results show that D3VO outperforms state-of-the-art traditional monocular VO methods by a large margin. It also achieves comparable results to state-of-the-art stereo/LiDAR odometry on KITTI and to the state-of-the-art visual-inertial odometry on EuRoC MAV, while using only a single camera.