# DefSLAM: Tracking and Mapping of Deforming Scenes from Monocular Sequences

arXiv: https://arxiv.org/abs/1908.08918 · HTML: https://mlsanity.com/paper/1908.08918/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Aug 2019.

## Built on by (most cited first, 30 of 37)

- [Endo-Depth-and-Motion: Reconstruction and Tracking in Endoscopic Videos using Depth Networks and Photometric Constraints](https://mlsanity.com/paper/2103.16525/) (2021)
- [E-DSSR: Efficient Dynamic Surgical Scene Reconstruction with Transformer-based Stereoscopic Depth Perception](https://mlsanity.com/paper/2107.00229/) (2021)
- [Tracking and Mapping in Medical Computer Vision: A Review](https://mlsanity.com/paper/2310.11475/) (2023)
- [SAGE: SLAM with Appearance and Geometry Prior for Endoscopy](https://mlsanity.com/paper/2202.09487/) (2022)
- [Recent Trends in 3D Reconstruction of General Non-Rigid Scenes](https://mlsanity.com/paper/2403.15064/) (2024)
- [Semantic-SuPer: A Semantic-aware Surgical Perception Framework for Endoscopic Tissue Identification, Reconstruction, and Tracking](https://mlsanity.com/paper/2210.16674/) (2022)
- [Event-based Non-Rigid Reconstruction from Contours](https://mlsanity.com/paper/2210.06270/) (2022)
- [SENDD: Sparse Efficient Neural Depth and Deformation for Tissue Tracking](https://mlsanity.com/paper/2305.06477/) (2023)
- [Photometric single-view dense 3D reconstruction in endoscopy](https://mlsanity.com/paper/2204.09083/) (2022)
- [Tracking monocular camera pose and deformation for SLAM inside the human body](https://mlsanity.com/paper/2204.08309/) (2022)
- [A geometry-aware deep network for depth estimation in monocular endoscopy](https://mlsanity.com/paper/2304.10241/) (2023)
- [Direct and Sparse Deformable Tracking](https://mlsanity.com/paper/2109.07370/) (2021)
- [ROBUSfT: Robust Real-Time Shape-from-Template, a C++ Library](https://mlsanity.com/paper/2301.04037/) (2023)
- [Real-time Nonrigid Mosaicking of Laparoscopy Images](https://mlsanity.com/paper/2103.07414/) (2021)
- [The Drunkard's Odometry: Estimating Camera Motion in Deforming Scenes](https://mlsanity.com/paper/2306.16917/) (2023)
- [BASED: Bundle-Adjusting Surgical Endoscopic Dynamic Video Reconstruction using Neural Radiance Fields](https://mlsanity.com/paper/2309.15329/) (2023)
- [Segment Anything for Videos: A Systematic Survey](https://mlsanity.com/paper/2408.08315/) (2024)
- [Ada-Tracker: Soft Tissue Tracking via Inter-Frame and Adaptive-Template Matching](https://mlsanity.com/paper/2403.06479/) (2024)
- [V3D-SLAM: Robust RGB-D SLAM in Dynamic Environments with 3D Semantic Geometry Voting](https://mlsanity.com/paper/2410.12068/) (2024)
- [EndoMetric: Near-Light Monocular Metric Scale Estimation in Endoscopy](https://mlsanity.com/paper/2410.15065/) (2024)
- [Deformable Surface Reconstruction via Riemannian Metric Preservation](https://mlsanity.com/paper/2212.11596/) (2022)
- [SuPerPM: A Surgical Perception Framework Based on Deep Point Matching Learned from Physical Constrained Simulation Data](https://mlsanity.com/paper/2309.13863/) (2023)
- [A Quantitative Evaluation of Dense 3D Reconstruction of Sinus Anatomy from Monocular Endoscopic Video](https://mlsanity.com/paper/2310.14364/) (2023)
- [BDIS-SLAM: A lightweight CPU-based dense stereo SLAM for surgery](https://mlsanity.com/paper/2312.15679/) (2023)
- [NRGS-SLAM: Monocular Non-Rigid SLAM for Endoscopy via Deformation-Aware 3D Gaussian Splatting](https://mlsanity.com/paper/2602.17182/) (2026)
- [Non-Rigid Structure-from-Motion via Differential Geometry with Recoverable Conformal Scale](https://mlsanity.com/paper/2510.01665/) (2025)
- [4D Monocular Surgical Reconstruction under Arbitrary Camera Motions](https://mlsanity.com/paper/2602.17473/) (2026)
- [3 Dimensional Dense Reconstruction: A Review of Algorithms and Dataset](https://mlsanity.com/paper/2304.09371/) (2023)
- [MultiViPerFrOG: A Globally Optimized Multi-Viewpoint Perception Framework for Camera Motion and Tissue Deformation](https://mlsanity.com/paper/2408.04367/) (2024)
- [Neural shape reconstruction from multiple views with static pattern projection](https://mlsanity.com/paper/2506.01389/) (2025)

## Abstract (arXiv)

Monocular SLAM algorithms perform robustly when observing rigid scenes, however, they fail when the observed scene deforms, for example, in medical endoscopy applications. We present DefSLAM, the first monocular SLAM capable of operating in deforming scenes in real-time. Our approach intertwines Shape-from-Template (SfT) and Non-Rigid Structure-from-Motion (NRSfM) techniques to deal with the exploratory sequences typical of SLAM. A deformation tracking thread recovers the pose of the camera and the deformation of the observed map, at frame rate, by means of SfT processing a template that models the scene shape-at-rest. A deformation mapping thread runs in parallel with the tracking to update the template, at keyframe rate, by means of an isometric NRSfM processing a batch of full perspective keyframes. In our experiments, DefSLAM processes close-up sequences of deforming scenes, both in a laboratory controlled experiment and in medical endoscopy sequences, producing accurate 3D models of the scene with respect to the moving camera.