# DeepFusion: Real-Time Dense 3D Reconstruction for Monocular SLAM using Single-View Depth and Gradient Predictions

arXiv: https://arxiv.org/abs/2207.12244 · HTML: https://mlsanity.com/paper/2207.12244/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 37 references that are in the map.
- First posted on arXiv 25 Jul 2022.

## Builds on (most cited first, 16 of 16)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras](https://mlsanity.com/paper/1610.06475/) (2016)
- [Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecture](https://mlsanity.com/paper/1411.4734/) (2014)
- [Deeper Depth Prediction with Fully Convolutional Residual Networks](https://mlsanity.com/paper/1606.00373/) (2016)
- [DeMoN: Depth and Motion Network for Learning Monocular Stereo](https://mlsanity.com/paper/1612.02401/) (2016)
- [CNN-SLAM: Real-time dense monocular SLAM with learned depth prediction](https://mlsanity.com/paper/1704.03489/) (2017)
- [CodeSLAM - Learning a Compact, Optimisable Representation for Dense Visual SLAM](https://mlsanity.com/paper/1804.00874/) (2018)
- [Deep Depth Completion of a Single RGB-D Image](https://mlsanity.com/paper/1803.09326/) (2018)
- [Deep Convolutional Neural Fields for Depth Estimation from a Single Image](https://mlsanity.com/paper/1411.6387/) (2014)
- [DeepTAM: Deep Tracking and Mapping](https://mlsanity.com/paper/1808.01900/) (2018)
- [Deep Virtual Stereo Odometry: Leveraging Deep Depth Prediction for Monocular Direct Sparse Odometry](https://mlsanity.com/paper/1807.02570/) (2018)
- [Single-View and Multi-View Depth Fusion](https://mlsanity.com/paper/1611.07245/) (2016)
- [Joint Prediction of Depths, Normals and Surface Curvature from RGB Images using CNNs](https://mlsanity.com/paper/1706.07593/) (2017)
- [Just-in-Time Reconstruction: Inpainting Sparse Maps using Single View Depth Predictors as Priors](https://mlsanity.com/paper/1805.04239/) (2018)

## Built on by (most cited first, 8 of 8)

- [The Reality Gap in Robotics: Challenges, Solutions, and Best Practices](https://mlsanity.com/paper/2510.20808/) (2025)
- [MACARONS: Mapping And Coverage Anticipation with RGB Online Self-Supervision](https://mlsanity.com/paper/2303.03315/) (2023)
- [DualCamCtrl: Dual-Branch Diffusion Model for Geometry-Aware Camera-Controlled Video Generation](https://mlsanity.com/paper/2511.23127/) (2025)
- [A Survey of 3D Reconstruction with Event Cameras](https://mlsanity.com/paper/2505.08438/) (2025)
- [ADU-Depth: Attention-based Distillation with Uncertainty Modeling for Depth Estimation](https://mlsanity.com/paper/2309.14744/) (2023)
- [Towards the Probabilistic Fusion of Learned Priors into Standard Pipelines for 3D Reconstruction](https://mlsanity.com/paper/2207.13464/) (2022)
- [DIME-Net: Neural Network-Based Dynamic Intrinsic Parameter Rectification for Cameras with Optical Image Stabilization System](https://mlsanity.com/paper/2303.11307/) (2023)
- [Knowledge representation and scalable abstract reasoning for simulated democracy in Unity](https://mlsanity.com/paper/2503.05783/) (2025)

## Abstract (arXiv)

While the keypoint-based maps created by sparse monocular simultaneous localisation and mapping (SLAM) systems are useful for camera tracking, dense 3D reconstructions may be desired for many robotic tasks. Solutions involving depth cameras are limited in range and to indoor spaces, and dense reconstruction systems based on minimising the photometric error between frames are typically poorly constrained and suffer from scale ambiguity. To address these issues, we propose a 3D reconstruction system that leverages the output of a convolutional neural network (CNN) to produce fully dense depth maps for keyframes that include metric scale. Our system, DeepFusion, is capable of producing real-time dense reconstructions on a GPU. It fuses the output of a semi-dense multiview stereo algorithm with the depth and gradient predictions of a CNN in a probabilistic fashion, using learned uncertainties produced by the network. While the network only needs to be run once per keyframe, we are able to optimise for the depth map with each new frame so as to constantly make use of new geometric constraints. Based on its performance on synthetic and real-world datasets, we demonstrate that DeepFusion is capable of performing at least as well as other comparable systems.