# CodeMapping: Real-Time Dense Mapping for Sparse SLAM using Compact Scene Representations

arXiv: https://arxiv.org/abs/2107.08994 · HTML: https://mlsanity.com/paper/2107.08994/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 28 references that are in the map.
- First posted on arXiv 19 Jul 2021.

## Builds on (most cited first, 15 of 15)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM](https://mlsanity.com/paper/2007.11898/) (2020)
- [Direct Sparse Odometry](https://mlsanity.com/paper/1607.02565/) (2016)
- [DeepMVS: Learning Multi-view Stereopsis](https://mlsanity.com/paper/1804.00650/) (2018)
- [D3VO: Deep Depth, Deep Pose and Deep Uncertainty for Monocular Visual Odometry](https://mlsanity.com/paper/2003.01060/) (2020)
- [Atlas: End-to-End 3D Scene Reconstruction from Posed Images](https://mlsanity.com/paper/2003.10432/) (2020)
- [Kimera: an Open-Source Library for Real-Time Metric-Semantic Localization and Mapping](https://mlsanity.com/paper/1910.02490/) (2019)
- [Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Image](https://mlsanity.com/paper/1709.07492/) (2017)
- [CNN-SLAM: Real-time dense monocular SLAM with learned depth prediction](https://mlsanity.com/paper/1704.03489/) (2017)
- [CodeSLAM - Learning a Compact, Optimisable Representation for Dense Visual SLAM](https://mlsanity.com/paper/1804.00874/) (2018)
- [DeepFactors: Real-Time Probabilistic Dense Monocular SLAM](https://mlsanity.com/paper/2001.05049/) (2020)
- [Deep Depth Completion of a Single RGB-D Image](https://mlsanity.com/paper/1803.09326/) (2018)
- [S3CNet: A Sparse Semantic Scene Completion Network for LiDAR Point Clouds](https://mlsanity.com/paper/2012.09242/) (2020)
- [CodeVIO: Visual-Inertial Odometry with Learned Optimizable Dense Depth](https://mlsanity.com/paper/2012.10133/) (2020)

## Built on by (most cited first, 16 of 16)

- [Vox-Fusion: Dense Tracking and Mapping with Voxel-based Neural Implicit Representation](https://mlsanity.com/paper/2210.15858/) (2022)
- [NeSLAM: Neural Implicit Mapping and Self-Supervised Feature Tracking With Depth Completion and Denoising](https://mlsanity.com/paper/2403.20034/) (2024)
- [iDF-SLAM: End-to-End RGB-D SLAM with Neural Implicit Mapping and Deep Feature Tracking](https://mlsanity.com/paper/2209.07919/) (2022)
- [MonoNeuralFusion: Online Monocular Neural 3D Reconstruction with Geometric Priors](https://mlsanity.com/paper/2209.15153/) (2022)
- [Struct-MDC: Mesh-Refined Unsupervised Depth Completion Leveraging Structural Regularities from Visual SLAM](https://mlsanity.com/paper/2204.13877/) (2022)
- [COMO: Compact Mapping and Odometry](https://mlsanity.com/paper/2404.03531/) (2024)
- [MGS-SLAM: Monocular Sparse Tracking and Gaussian Mapping with Depth Smooth Regularization](https://mlsanity.com/paper/2405.06241/) (2024)
- [GeoRefine: Self-Supervised Online Depth Refinement for Accurate Dense Mapping](https://mlsanity.com/paper/2205.01656/) (2022)
- [Vox-Fusion++: Voxel-based Neural Implicit Dense Tracking and Mapping with Multi-maps](https://mlsanity.com/paper/2403.12536/) (2024)
- [SimpleMapping: Real-Time Visual-Inertial Dense Mapping with Deep Multi-View Stereo](https://mlsanity.com/paper/2306.08648/) (2023)
- [ProvNeRF: Modeling per Point Provenance in NeRFs as a Stochastic Field](https://mlsanity.com/paper/2401.08140/) (2024)
- [EGG-Fusion: Efficient 3D Reconstruction with Geometry-aware Gaussian Surfel on the Fly](https://mlsanity.com/paper/2512.01296/) (2025)
- [Depth Completion with Multiple Balanced Bases and Confidence for Dense Monocular SLAM](https://mlsanity.com/paper/2309.04145/) (2023)
- [4D Primitive-Mach'e: Glueing Primitives for Persistent 4D Scene Reconstruction](https://mlsanity.com/paper/2512.16564/) (2025)
- [CSMapping: Scalable Crowdsourced Semantic Mapping and Topology Inference for Autonomous Driving](https://mlsanity.com/paper/2512.03510/) (2025)
- [PCR-ORB: Enhanced ORB-SLAM3 with Point Cloud Refinement Using Deep Learning-Based Dynamic Object Filtering](https://mlsanity.com/paper/2512.23318/) (2025)

## Abstract (arXiv)

We propose a novel dense mapping framework for sparse visual SLAM systems which leverages a compact scene representation. State-of-the-art sparse visual SLAM systems provide accurate and reliable estimates of the camera trajectory and locations of landmarks. While these sparse maps are useful for localization, they cannot be used for other tasks such as obstacle avoidance or scene understanding. In this paper we propose a dense mapping framework to complement sparse visual SLAM systems which takes as input the camera poses, keyframes and sparse points produced by the SLAM system and predicts a dense depth image for every keyframe. We build on CodeSLAM and use a variational autoencoder (VAE) which is conditioned on intensity, sparse depth and reprojection error images from sparse SLAM to predict an uncertainty-aware dense depth map. The use of a VAE then enables us to refine the dense depth images through multi-view optimization which improves the consistency of overlapping frames. Our mapper runs in a separate thread in parallel to the SLAM system in a loosely coupled manner. This flexible design allows for integration with arbitrary metric sparse SLAM systems without delaying the main SLAM process. Our dense mapper can be used not only for local mapping but also globally consistent dense 3D reconstruction through TSDF fusion. We demonstrate our system running with ORB-SLAM3 and show accurate dense depth estimation which could enable applications such as robotics and augmented reality.