# iDF-SLAM: End-to-End RGB-D SLAM with Neural Implicit Mapping and Deep Feature Tracking

arXiv: https://arxiv.org/abs/2209.07919 · HTML: https://mlsanity.com/paper/2209.07919/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 35 references that are in the map.
- First posted on arXiv 16 Sep 2022.

## Builds on (most cited first, 27 of 27)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [Neural Sparse Voxel Fields](https://mlsanity.com/paper/2007.11571/) (2020)
- [The Replica Dataset: A Digital Replica of Indoor Spaces](https://mlsanity.com/paper/1906.05797/) (2019)
- [UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction](https://mlsanity.com/paper/2104.10078/) (2021)
- [BARF: Bundle-Adjusting Neural Radiance Fields](https://mlsanity.com/paper/2104.06405/) (2021)
- [NICE-SLAM: Neural Implicit Scalable Encoding for SLAM](https://mlsanity.com/paper/2112.12130/) (2021)
- [iMAP: Implicit Mapping and Positioning in Real-Time](https://mlsanity.com/paper/2103.12352/) (2021)
- [INeRF: Inverting Neural Radiance Fields for Pose Estimation](https://mlsanity.com/paper/2012.05877/) (2020)
- [Neural RGB-D Surface Reconstruction](https://mlsanity.com/paper/2104.04532/) (2021)
- [Deep Global Registration](https://mlsanity.com/paper/2004.11540/) (2020)
- [CodeSLAM - Learning a Compact, Optimisable Representation for Dense Visual SLAM](https://mlsanity.com/paper/1804.00874/) (2018)
- [SemanticFusion: Dense 3D Semantic Mapping with Convolutional Neural Networks](https://mlsanity.com/paper/1609.05130/) (2016)
- [DeepFactors: Real-Time Probabilistic Dense Monocular SLAM](https://mlsanity.com/paper/2001.05049/) (2020)
- [UnsupervisedR&R: Unsupervised Point Cloud Registration via Differentiable Rendering](https://mlsanity.com/paper/2102.11870/) (2021)
- [SceneCode: Monocular Dense Semantic Reconstruction using Learned Encoded Scene Representations](https://mlsanity.com/paper/1903.06482/) (2019)
- [Meaningful Maps With Object-Oriented Semantic Mapping](https://mlsanity.com/paper/1609.07849/) (2016)
- [PVO: Panoptic Visual Odometry](https://mlsanity.com/paper/2207.01610/) (2022)
- [FD-SLAM: 3-D Reconstruction Using Features and Dense Matching](https://mlsanity.com/paper/2203.13861/) (2022)
- [CodeMapping: Real-Time Dense Mapping for Sparse SLAM using Compact Scene Representations](https://mlsanity.com/paper/2107.08994/) (2021)
- [NeuralBlox: Real-Time Neural Representation Fusion for Robust Volumetric Mapping](https://mlsanity.com/paper/2110.09415/) (2021)
- [CGiS-Net: Aggregating Colour, Geometry and Implicit Semantic Features for Indoor Place Recognition](https://mlsanity.com/paper/2202.02070/) (2022)
- [D3FlowSLAM: Self-Supervised Dynamic SLAM with Flow Motion Decomposition and DINO Guidance](https://mlsanity.com/paper/2207.08794/) (2022)
- [Object-Augmented RGB-D SLAM for Wide-Disparity Relocalisation](https://mlsanity.com/paper/2108.02522/) (2021)

## Built on by (most cited first, 22 of 22)

- [SplaTAM: Splat, Track & Map 3D Gaussians for Dense RGB-D SLAM](https://mlsanity.com/paper/2312.02126/) (2023)
- [PGSR: Planar-based Gaussian Splatting for Efficient and High-Fidelity Surface Reconstruction](https://mlsanity.com/paper/2406.06521/) (2024)
- [NICER-SLAM: Neural Implicit Scene Encoding for RGB SLAM](https://mlsanity.com/paper/2302.03594/) (2023)
- [ESLAM: Efficient Dense SLAM System Based on Hybrid Representation of Signed Distance Fields](https://mlsanity.com/paper/2211.11704/) (2022)
- [How NeRFs and 3D Gaussian Splatting are Reshaping SLAM: a Survey](https://mlsanity.com/paper/2402.13255/) (2024)
- [IntrinsicNeRF: Learning Intrinsic Neural Radiance Fields for Editable Novel View Synthesis](https://mlsanity.com/paper/2210.00647/) (2022)
- [DiffPano: Scalable and Consistent Text to Panorama Generation with Spherical Epipolar-Aware Diffusion](https://mlsanity.com/paper/2410.24203/) (2024)
- [PhysFlow: Unleashing the Potential of Multi-modal Foundation Models and Video Diffusion for 4D Dynamic Physical Scene Simulation](https://mlsanity.com/paper/2411.14423/) (2024)
- [CityGS-X: A Scalable Architecture for Efficient and Geometrically Accurate Large-Scale Scene Reconstruction](https://mlsanity.com/paper/2503.23044/) (2025)
- [MIPS-Fusion: Multi-Implicit-Submaps for Scalable and Robust Online Neural RGB-D Reconstruction](https://mlsanity.com/paper/2308.08741/) (2023)
- [DDN-SLAM: Real-time Dense Dynamic Neural Implicit SLAM](https://mlsanity.com/paper/2401.01545/) (2024)
- [GigaGS: Scaling up Planar-Based 3D Gaussians for Large Scene Surface Reconstruction](https://mlsanity.com/paper/2409.06685/) (2024)
- [StreetSurfGS: Scalable Urban Street Surface Reconstruction with Planar-based Gaussian Splatting](https://mlsanity.com/paper/2410.04354/) (2024)
- [NeRFs in Robotics: A Survey](https://mlsanity.com/paper/2405.01333/) (2024)
- [NeuRodin: A Two-stage Framework for High-Fidelity Neural Surface Reconstruction](https://mlsanity.com/paper/2408.10178/) (2024)
- [TSGS: Improving Gaussian Splatting for Transparent Surface Reconstruction via Normal and De-lighting Priors](https://mlsanity.com/paper/2504.12799/) (2025)
- [ND-SDF: Learning Normal Deflection Fields for High-Fidelity Indoor Reconstruction](https://mlsanity.com/paper/2408.12598/) (2024)
- [FMapping: Factorized Efficient Neural Field Mapping for Real-Time Dense RGB SLAM](https://mlsanity.com/paper/2306.00579/) (2023)
- [Learn to Memorize and to Forget: A Continual Learning Perspective of Dynamic SLAM](https://mlsanity.com/paper/2407.13338/) (2024)
- [CAD-SLAM: Consistency-Aware Dynamic SLAM with Dynamic-Static Decoupled Mapping](https://mlsanity.com/paper/2505.19420/) (2025)
- [HVOFusion: Incremental Mesh Reconstruction Using Hybrid Voxel Octree](https://mlsanity.com/paper/2404.17974/) (2024)
- [Deep Learning-Powered Visual SLAM Aimed at Assisting Visually Impaired Navigation](https://mlsanity.com/paper/2510.20549/) (2025)

## Abstract (arXiv)

We propose a novel end-to-end RGB-D SLAM, iDF-SLAM, which adopts a feature-based deep neural tracker as the front-end and a NeRF-style neural implicit mapper as the back-end. The neural implicit mapper is trained on-the-fly, while though the neural tracker is pretrained on the ScanNet dataset, it is also finetuned along with the training of the neural implicit mapper. Under such a design, our iDF-SLAM is capable of learning to use scene-specific features for camera tracking, thus enabling lifelong learning of the SLAM system. Both the training for the tracker and the mapper are self-supervised without introducing ground truth poses. We test the performance of our iDF-SLAM on the Replica and ScanNet datasets and compare the results to the two recent NeRF-based neural SLAM systems. The proposed iDF-SLAM demonstrates state-of-the-art results in terms of scene reconstruction and competitive performance in camera tracking.