# MIPS-Fusion: Multi-Implicit-Submaps for Scalable and Robust Online Neural RGB-D Reconstruction

arXiv: https://arxiv.org/abs/2308.08741 · HTML: https://mlsanity.com/paper/2308.08741/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 73 references that are in the map.
- First posted on arXiv 17 Aug 2023.

## Builds on (most cited first, 30 of 37)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction](https://mlsanity.com/paper/2111.11215/) (2021)
- [Neural Sparse Voxel Fields](https://mlsanity.com/paper/2007.11571/) (2020)
- [The Replica Dataset: A Digital Replica of Indoor Spaces](https://mlsanity.com/paper/1906.05797/) (2019)
- [Convolutional Occupancy Networks](https://mlsanity.com/paper/2003.04618/) (2020)
- [Block-NeRF: Scalable Large Scene Neural View Synthesis](https://mlsanity.com/paper/2202.05263/) (2022)
- [DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Cameras](https://mlsanity.com/paper/2108.10869/) (2021)
- [NICE-SLAM: Neural Implicit Scalable Encoding for SLAM](https://mlsanity.com/paper/2112.12130/) (2021)
- [iMAP: Implicit Mapping and Positioning in Real-Time](https://mlsanity.com/paper/2103.12352/) (2021)
- [Neural RGB-D Surface Reconstruction](https://mlsanity.com/paper/2104.04532/) (2021)
- [NeRF-SLAM: Real-Time Dense Monocular SLAM with Neural Radiance Fields](https://mlsanity.com/paper/2210.13641/) (2022)
- [Co-SLAM: Joint Coordinate and Sparse Parametric Encodings for Neural Real-Time SLAM](https://mlsanity.com/paper/2304.14377/) (2023)
- [BA-Net: Dense Bundle Adjustment Network](https://mlsanity.com/paper/1806.04807/) (2018)
- [ESLAM: Efficient Dense SLAM System Based on Hybrid Representation of Signed Distance Fields](https://mlsanity.com/paper/2211.11704/) (2022)
- [CodeSLAM - Learning a Compact, Optimisable Representation for Dense Visual SLAM](https://mlsanity.com/paper/1804.00874/) (2018)
- [Vox-Fusion: Dense Tracking and Mapping with Voxel-based Neural Implicit Representation](https://mlsanity.com/paper/2210.15858/) (2022)
- [NeSF: Neural Semantic Fields for Generalizable Semantic Segmentation of 3D Scenes](https://mlsanity.com/paper/2111.13260/) (2021)
- [BundleFusion: Real-time Globally Consistent 3D Reconstruction using On-the-fly Surface Re-integration](https://mlsanity.com/paper/1604.01093/) (2016)
- [iSDF: Real-Time Neural Signed Distance Fields for Robot Perception](https://mlsanity.com/paper/2204.02296/) (2022)
- [Orbeez-SLAM: A Real-time Monocular Visual SLAM with ORB Features and NeRF-realized Mapping](https://mlsanity.com/paper/2209.13274/) (2022)
- [Probing Representation Forgetting in Supervised and Unsupervised Continual Learning](https://mlsanity.com/paper/2203.13381/) (2022)
- [DI-Fusion: Online Implicit 3D Reconstruction with Deep Priors](https://mlsanity.com/paper/2012.05551/) (2020)
- [RoutedFusion: Learning Real-time Depth Map Fusion](https://mlsanity.com/paper/2001.04388/) (2020)
- [Vox-Surf: Voxel-based Implicit Surface Representation](https://mlsanity.com/paper/2208.10925/) (2022)
- [NeuralFusion: Online Depth Fusion in Latent Space](https://mlsanity.com/paper/2011.14791/) (2020)
- [Fusion-Aware Point Convolution for Online Semantic 3D Scene Segmentation](https://mlsanity.com/paper/2003.06233/) (2020)
- [TANDEM: Tracking and Dense Mapping in Real-time using Deep Multi-view Stereo](https://mlsanity.com/paper/2111.07418/) (2021)
- [Lightweight Unsupervised Deep Loop Closure](https://mlsanity.com/paper/1805.07703/) (2018)

## Built on by (most cited first, 24 of 24)

- [Gaussian-SLAM: Photo-realistic Dense SLAM with Gaussian Splatting](https://mlsanity.com/paper/2312.10070/) (2023)
- [How NeRFs and 3D Gaussian Splatting are Reshaping SLAM: a Survey](https://mlsanity.com/paper/2402.13255/) (2024)
- [Splat-SLAM: Globally Optimized RGB-only SLAM with 3D Gaussians](https://mlsanity.com/paper/2405.16544/) (2024)
- [LoopSplat: Loop Closure by Registering 3D Gaussian Splats](https://mlsanity.com/paper/2408.10154/) (2024)
- [Loopy-SLAM: Dense Neural SLAM with Loop Closures](https://mlsanity.com/paper/2402.09944/) (2024)
- [PIN-SLAM: LiDAR SLAM Using a Point-Based Implicit Neural Representation for Achieving Global Map Consistency](https://mlsanity.com/paper/2401.09101/) (2024)
- [GlORIE-SLAM: Globally Optimized RGB-only Implicit Encoding Point Cloud SLAM](https://mlsanity.com/paper/2403.19549/) (2024)
- [Neural Radiance Field in Autonomous Driving: A Survey](https://mlsanity.com/paper/2404.13816/) (2024)
- [DenseSplat: Densifying Gaussian Splatting SLAM with Neural Radiance Prior](https://mlsanity.com/paper/2502.09111/) (2025)
- [MCN-SLAM: Multi-Agent Collaborative Neural SLAM with Hybrid Implicit Neural Scene Representation](https://mlsanity.com/paper/2506.18678/) (2025)
- [Hi-Map: Hierarchical Factorized Radiance Field for High-Fidelity Monocular Dense Mapping](https://mlsanity.com/paper/2401.03203/) (2024)
- [Benchmarking Implicit Neural Representation and Geometric Rendering in Real-Time RGB-D SLAM](https://mlsanity.com/paper/2403.19473/) (2024)
- [Neural Graph Map: Dense Mapping with Efficient Loop Closure Integration](https://mlsanity.com/paper/2405.03633/) (2024)
- [Uni-SLAM: Uncertainty-Aware Neural Implicit SLAM for Real-Time Dense Indoor Scene Reconstruction](https://mlsanity.com/paper/2412.00242/) (2024)
- [BoxFusion: Reconstruction-Free Open-Vocabulary 3D Object Detection via Real-Time Multi-View Box Fusion](https://mlsanity.com/paper/2506.15610/) (2025)
- [VarSplat: Uncertainty-aware 3D Gaussian Splatting for Robust RGB-D SLAM](https://mlsanity.com/paper/2603.09673/) (2026)
- [NeB-SLAM: Neural Blocks-based Salable RGB-D SLAM for Unknown Scenes](https://mlsanity.com/paper/2405.15151/) (2024)
- [Deblur Gaussian Splatting SLAM](https://mlsanity.com/paper/2503.12572/) (2025)
- [RemixFusion: Residual-based Mixed Representation for Large-scale Online RGB-D Reconstruction](https://mlsanity.com/paper/2507.17594/) (2025)
- [F2M-Reg: Unsupervised RGB-D Point Cloud Registration with Frame-to-Model Optimization](https://mlsanity.com/paper/2405.00507/) (2024)
- [ConfidentSplat: Confidence-Weighted Depth Fusion for Accurate 3D Gaussian Splatting SLAM](https://mlsanity.com/paper/2509.16863/) (2025)
- [PROFusion: Robust and Accurate Dense Reconstruction via Camera Pose Regression and Optimization](https://mlsanity.com/paper/2509.24236/) (2025)
- [LatentAM: Real-Time, Large-Scale Latent Gaussian Attention Mapping via Online Dictionary Learning](https://mlsanity.com/paper/2602.12314/) (2026)
- [HIGS: Hierarchical Implicit Grids for Joint Geometric and Semantic Scene Understanding](https://mlsanity.com/paper/2609.38620/) (2026)

## Abstract (arXiv)

We introduce MIPS-Fusion, a robust and scalable online RGB-D reconstruction method based on a novel neural implicit representation -- multi-implicit-submap. Different from existing neural RGB-D reconstruction methods lacking either flexibility with a single neural map or scalability due to extra storage of feature grids, we propose a pure neural representation tackling both difficulties with a divide-and-conquer design. In our method, neural submaps are incrementally allocated alongside the scanning trajectory and efficiently learned with local neural bundle adjustments. The submaps can be refined individually in a back-end optimization and optimized jointly to realize submap-level loop closure. Meanwhile, we propose a hybrid tracking approach combining randomized and gradient-based pose optimizations. For the first time, randomized optimization is made possible in neural tracking with several key designs to the learning process, enabling efficient and robust tracking even under fast camera motions. The extensive evaluation demonstrates that our method attains higher reconstruction quality than the state of the arts for large-scale scenes and under fast camera motions.