# gradSLAM: Automagically differentiable SLAM

arXiv: https://arxiv.org/abs/1910.10672 · HTML: https://mlsanity.com/paper/1910.10672/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Oct 2019.

## Built on by (most cited first, 27 of 27)

- [DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Cameras](https://mlsanity.com/paper/2108.10869/) (2021)
- [ConceptFusion: Open-set Multimodal 3D Mapping](https://mlsanity.com/paper/2302.07241/) (2023)
- [VGGT-SLAM: Dense RGB SLAM Optimized on the SL(4) Manifold](https://mlsanity.com/paper/2505.12549/) (2025)
- [Theseus: A Library for Differentiable Nonlinear Optimization](https://mlsanity.com/paper/2207.09442/) (2022)
- [ViKiNG: Vision-Based Kilometer-Scale Navigation with Geographic Hints](https://mlsanity.com/paper/2202.11271/) (2022)
- [Tangent Space Backpropagation for 3D Transformation Groups](https://mlsanity.com/paper/2103.12032/) (2021)
- [Efficient Differentiable Simulation of Articulated Bodies](https://mlsanity.com/paper/2109.07719/) (2021)
- [TANDEM: Tracking and Dense Mapping in Real-time using Deep Multi-view Stereo](https://mlsanity.com/paper/2111.07418/) (2021)
- [FlowCam: Training Generalizable 3D Radiance Fields without Camera Poses via Pixel-Aligned Scene Flow](https://mlsanity.com/paper/2306.00180/) (2023)
- [DiffStack: A Differentiable and Modular Control Stack for Autonomous Vehicles](https://mlsanity.com/paper/2212.06437/) (2022)
- [Differentiable SLAM-net: Learning Particle SLAM for Visual Navigation](https://mlsanity.com/paper/2105.07593/) (2021)
- [Offline Reinforcement Learning for Visual Navigation](https://mlsanity.com/paper/2212.08244/) (2022)
- [Advances in Inference and Representation for Simultaneous Localization and Mapping](https://mlsanity.com/paper/2103.05041/) (2021)
- [Language-EXtended Indoor SLAM (LEXIS): A Versatile System for Real-time Visual Scene Understanding](https://mlsanity.com/paper/2309.15065/) (2023)
- [Learning Correspondence Uncertainty via Differentiable Nonlinear Least Squares](https://mlsanity.com/paper/2305.09527/) (2023)
- [Learning Robotic Navigation from Experience: Principles, Methods, and Recent Results](https://mlsanity.com/paper/2212.06759/) (2022)
- [RTG-SLAM: Real-time 3D Reconstruction at Scale using Gaussian Splatting](https://mlsanity.com/paper/2404.19706/) (2024)
- [Bundle Adjustment in the Eager Mode](https://mlsanity.com/paper/2409.12190/) (2024)
- [RAG-3DSG: Enhancing 3D Scene Graphs with Re-Shot Guided Retrieval-Augmented Generation](https://mlsanity.com/paper/2601.10168/) (2026)
- [Monocular visual simultaneous localization and mapping: (r)evolution from geometry to deep learning-based pipelines](https://mlsanity.com/paper/2503.02955/) (2025)
- [G3T Up! Gravity Aligned Coordinate Frames Simplify Pointmap Processing](https://mlsanity.com/paper/2605.27372/) (2026)
- [Differentiable SLAM Helps Deep Learning-based LiDAR Perception Tasks](https://mlsanity.com/paper/2309.09206/) (2023)
- [From Variance to Veracity: Unbundling and Mitigating Gradient Variance in Differentiable Bundle Adjustment Layers](https://mlsanity.com/paper/2406.07785/) (2024)
- [Direction-aware 3D Large Multimodal Models](https://mlsanity.com/paper/2602.19063/) (2026)
- [Compositional Semantics for Open Vocabulary Spatio-semantic Representations](https://mlsanity.com/paper/2310.04981/) (2023)
- [Lost in Tracking Translation: A Comprehensive Analysis of Visual SLAM in Human-Centered XR and IoT Ecosystems](https://mlsanity.com/paper/2411.07146/) (2024)
- [PickScan: Object discovery and reconstruction from handheld interactions](https://mlsanity.com/paper/2411.11196/) (2024)

## Abstract (arXiv)

Blending representation learning approaches with simultaneous localization and mapping (SLAM) systems is an open question, because of their highly modular and complex nature. Functionally, SLAM is an operation that transforms raw sensor inputs into a distribution over the state(s) of the robot and the environment. If this transformation (SLAM) were expressible as a differentiable function, we could leverage task-based error signals to learn representations that optimize task performance. However, several components of a typical dense SLAM system are non-differentiable. In this work, we propose gradSLAM, a methodology for posing SLAM systems as differentiable computational graphs, which unifies gradient-based learning and SLAM. We propose differentiable trust-region optimizers, surface measurement and fusion schemes, and raycasting, without sacrificing accuracy. This amalgamation of dense SLAM with computational graphs enables us to backprop all the way from 3D maps to 2D pixels, opening up new possibilities in gradient-based learning for SLAM. TL;DR: We leverage the power of automatic differentiation frameworks to make dense SLAM differentiable.