# Vox-Fusion: Dense Tracking and Mapping with Voxel-based Neural Implicit Representation

arXiv: https://arxiv.org/abs/2210.15858 · HTML: https://mlsanity.com/paper/2210.15858/

## Key facts

- Cited by 149 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 41 references that are in the map.
- First posted on arXiv 28 Oct 2022.

## Builds on (most cited first, 23 of 23)

- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [Plenoxels: Radiance Fields without Neural Networks](https://mlsanity.com/paper/2112.05131/) (2021)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [Neural Sparse Voxel Fields](https://mlsanity.com/paper/2007.11571/) (2020)
- [The Replica Dataset: A Digital Replica of Indoor Spaces](https://mlsanity.com/paper/1906.05797/) (2019)
- [ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM](https://mlsanity.com/paper/2007.11898/) (2020)
- [ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras](https://mlsanity.com/paper/1610.06475/) (2016)
- [DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Cameras](https://mlsanity.com/paper/2108.10869/) (2021)
- [UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction](https://mlsanity.com/paper/2104.10078/) (2021)
- [NICE-SLAM: Neural Implicit Scalable Encoding for SLAM](https://mlsanity.com/paper/2112.12130/) (2021)
- [iMAP: Implicit Mapping and Positioning in Real-Time](https://mlsanity.com/paper/2103.12352/) (2021)
- [Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapes](https://mlsanity.com/paper/2101.10994/) (2021)
- [Neural RGB-D Surface Reconstruction](https://mlsanity.com/paper/2104.04532/) (2021)
- [Urban Radiance Fields](https://mlsanity.com/paper/2111.14643/) (2021)
- [CodeSLAM - Learning a Compact, Optimisable Representation for Dense Visual SLAM](https://mlsanity.com/paper/1804.00874/) (2018)
- [DeepFactors: Real-Time Probabilistic Dense Monocular SLAM](https://mlsanity.com/paper/2001.05049/) (2020)
- [DI-Fusion: Online Implicit 3D Reconstruction with Deep Priors](https://mlsanity.com/paper/2012.05551/) (2020)
- [Vox-Surf: Voxel-based Implicit Surface Representation](https://mlsanity.com/paper/2208.10925/) (2022)
- [Real-time Scalable Dense Surfel Mapping](https://mlsanity.com/paper/1909.04250/) (2019)
- [FD-SLAM: 3-D Reconstruction Using Features and Dense Matching](https://mlsanity.com/paper/2203.13861/) (2022)
- [CodeMapping: Real-Time Dense Mapping for Sparse SLAM using Compact Scene Representations](https://mlsanity.com/paper/2107.08994/) (2021)
- [Learning Deeply Supervised Good Features to Match for Dense Monocular Reconstruction](https://mlsanity.com/paper/1711.05919/) (2017)

## Built on by (most cited first, 30 of 149)

- [Gaussian Splatting SLAM](https://mlsanity.com/paper/2312.06741/) (2023)
- [SplaTAM: Splat, Track & Map 3D Gaussians for Dense RGB-D SLAM](https://mlsanity.com/paper/2312.02126/) (2023)
- [GS-SLAM: Dense Visual SLAM with 3D Gaussian Splatting](https://mlsanity.com/paper/2311.11700/) (2023)
- [A Survey on 3D Gaussian Splatting](https://mlsanity.com/paper/2401.03890/) (2024)
- [NICER-SLAM: Neural Implicit Scene Encoding for RGB SLAM](https://mlsanity.com/paper/2302.03594/) (2023)
- [Point-SLAM: Dense Neural Point Cloud-based SLAM](https://mlsanity.com/paper/2304.04278/) (2023)
- [Gaussian-SLAM: Photo-realistic Dense SLAM with Gaussian Splatting](https://mlsanity.com/paper/2312.10070/) (2023)
- [Recent Advances in 3D Gaussian Splatting](https://mlsanity.com/paper/2403.11134/) (2024)
- [Progressively Optimized Local Radiance Fields for Robust View Synthesis](https://mlsanity.com/paper/2303.13791/) (2023)
- [SINE: Semantic-driven Image-based NeRF Editing with Prior-guided Editing Field](https://mlsanity.com/paper/2303.13277/) (2023)
- [SGS-SLAM: Semantic Gaussian Splatting For Neural Dense SLAM](https://mlsanity.com/paper/2402.03246/) (2024)
- [How NeRFs and 3D Gaussian Splatting are Reshaping SLAM: a Survey](https://mlsanity.com/paper/2402.13255/) (2024)
- [vMAP: Vectorised Object Mapping for Neural Field SLAM](https://mlsanity.com/paper/2302.01838/) (2023)
- [Splat-SLAM: Globally Optimized RGB-only SLAM with 3D Gaussians](https://mlsanity.com/paper/2405.16544/) (2024)
- [RGBD GS-ICP SLAM](https://mlsanity.com/paper/2403.12550/) (2024)
- [PLGSLAM: Progressive Neural Scene Represenation with Local to Global Bundle Adjustment](https://mlsanity.com/paper/2312.09866/) (2023)
- [CG-SLAM: Efficient Dense RGB-D SLAM in a Consistent Uncertainty-aware 3D Gaussian Field](https://mlsanity.com/paper/2403.16095/) (2024)
- [LoopSplat: Loop Closure by Registering 3D Gaussian Splats](https://mlsanity.com/paper/2408.10154/) (2024)
- [GeoGaussian: Geometry-aware Gaussian Splatting for Scene Rendering](https://mlsanity.com/paper/2403.11324/) (2024)
- [Loopy-SLAM: Dense Neural SLAM with Loop Closures](https://mlsanity.com/paper/2402.09944/) (2024)
- [SplatLoc: 3D Gaussian Splatting-based Visual Localization for Augmented Reality](https://mlsanity.com/paper/2409.14067/) (2024)
- [Dense RGB SLAM with Neural Implicit Maps](https://mlsanity.com/paper/2301.08930/) (2023)
- [PIN-SLAM: LiDAR SLAM Using a Point-Based Implicit Neural Representation for Achieving Global Map Consistency](https://mlsanity.com/paper/2401.09101/) (2024)
- [SemGauss-SLAM: Dense Semantic Gaussian Splatting SLAM](https://mlsanity.com/paper/2403.07494/) (2024)
- [NeRF-LOAM: Neural Implicit Representation for Large-Scale Incremental LiDAR Odometry and Mapping](https://mlsanity.com/paper/2303.10709/) (2023)
- [GlORIE-SLAM: Globally Optimized RGB-only Implicit Encoding Point Cloud SLAM](https://mlsanity.com/paper/2403.19549/) (2024)
- [PATS: Patch Area Transportation with Subdivision for Local Feature Matching](https://mlsanity.com/paper/2303.07700/) (2023)
- [CP-SLAM: Collaborative Neural Point-based SLAM System](https://mlsanity.com/paper/2311.08013/) (2023)
- [HI-SLAM: Monocular Real-time Dense Mapping with Hybrid Implicit Fields](https://mlsanity.com/paper/2310.04787/) (2023)
- [Neural Implicit Dense Semantic SLAM](https://mlsanity.com/paper/2304.14560/) (2023)

## Abstract (arXiv)

In this work, we present a dense tracking and mapping system named Vox-Fusion, which seamlessly fuses neural implicit representations with traditional volumetric fusion methods. Our approach is inspired by the recently developed implicit mapping and positioning system and further extends the idea so that it can be freely applied to practical scenarios. Specifically, we leverage a voxel-based neural implicit surface representation to encode and optimize the scene inside each voxel. Furthermore, we adopt an octree-based structure to divide the scene and support dynamic expansion, enabling our system to track and map arbitrary scenes without knowing the environment like in previous works. Moreover, we proposed a high-performance multi-process framework to speed up the method, thus supporting some applications that require real-time performance. The evaluation results show that our methods can achieve better accuracy and completeness than previous methods. We also show that our Vox-Fusion can be used in augmented reality and virtual reality applications. Our source code is publicly available at https://github.com/zju3dv/Vox-Fusion.