# SLAM assisted 3D tracking system for laparoscopic surgery

arXiv: https://arxiv.org/abs/2409.11688 · HTML: https://mlsanity.com/paper/2409.11688/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 38 references that are in the map.
- First posted on arXiv 18 Sep 2024.

## Builds on (most cited first, 9 of 9)

- [ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras](https://mlsanity.com/paper/1610.06475/) (2016)
- [Deep Learning for 3D Point Clouds: A Survey](https://mlsanity.com/paper/1912.12033/) (2019)
- [A Survey of Non-Rigid 3D Registration](https://mlsanity.com/paper/2203.07858/) (2022)
- [Learning How To Robustly Estimate Camera Pose in Endoscopic Videos](https://mlsanity.com/paper/2304.08023/) (2023)
- [MIS-SLAM: Real-time Large Scale Dense Deformable SLAM System in Minimal Invasive Surgery Based on Heterogeneous Computing](https://mlsanity.com/paper/1803.02009/) (2018)
- [Convex Relaxations for Pose Graph Optimization with Outliers](https://mlsanity.com/paper/1801.02112/) (2018)
- [Autonomous Navigation for Robot-assisted Intraluminal and Endovascular Procedures: A Systematic Review](https://mlsanity.com/paper/2305.04027/) (2023)
- [RIDE: Self-Supervised Learning of Rotation-Equivariant Keypoint Detection and Invariant Description for Endoscopy](https://mlsanity.com/paper/2309.09563/) (2023)
- [Combining Deep Learning with Geometric Features for Image based Localization in the Gastrointestinal Tract](https://mlsanity.com/paper/2005.05481/) (2020)

## Built on by (most cited first, 2 of 2)

- [Towards Next-Generation Healthcare: A Survey of Medical Embodied AI for Perception, Decision-Making, and Action](https://mlsanity.com/paper/2606.15647/) (2026)
- [LapaTrack-3D: 6 DoF pre-operative shape tracking for laparoscopic surgery](https://mlsanity.com/paper/2609.19954/) (2026)

## Abstract (arXiv)

A major limitation of minimally invasive surgery is the difficulty in accurately locating the internal anatomical structures of the target organ due to the lack of tactile feedback and transparency. Augmented reality (AR) offers a promising solution to overcome this challenge. Numerous studies have shown that combining learning-based and geometric methods can achieve accurate preoperative and intraoperative data registration. This work proposes a real-time monocular 3D tracking algorithm for post-registration tasks. The ORB-SLAM2 framework is adopted and modified for prior-based 3D tracking. The primitive 3D shape is used for fast initialization of the monocular SLAM. A pseudo-segmentation strategy is employed to separate the target organ from the background for tracking purposes, and the geometric prior of the 3D shape is incorporated as an additional constraint in the pose graph. Experiments from in-vivo and ex-vivo tests demonstrate that the proposed 3D tracking system provides robust 3D tracking and effectively handles typical challenges such as fast motion, out-of-field-of-view scenarios, partial visibility, and "organ-background" relative motion.