# PERSEUS: Perception with Semantic Endoscopic Understanding and SLAM

arXiv: https://arxiv.org/abs/2509.13541 · HTML: https://mlsanity.com/paper/2509.13541/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 38 references that are in the map.
- First posted on arXiv 16 Sep 2025.

## Builds on (most cited first, 13 of 13)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [Depth Anything V2](https://mlsanity.com/paper/2406.09414/) (2024)
- [DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Cameras](https://mlsanity.com/paper/2108.10869/) (2021)
- [LightGlue: Local Feature Matching at Light Speed](https://mlsanity.com/paper/2306.13643/) (2023)
- [From Coarse to Fine: Robust Hierarchical Localization at Large Scale](https://mlsanity.com/paper/1812.03506/) (2018)
- [DISK: Learning local features with policy gradient](https://mlsanity.com/paper/2006.13566/) (2020)
- [SAGE: SLAM with Appearance and Geometry Prior for Endoscopy](https://mlsanity.com/paper/2202.09487/) (2022)
- [Semantic-SuPer: A Semantic-aware Surgical Perception Framework for Endoscopic Tissue Identification, Reconstruction, and Tracking](https://mlsanity.com/paper/2210.16674/) (2022)
- [Promise:Prompt-driven 3D Medical Image Segmentation Using Pretrained Image Foundation Models](https://mlsanity.com/paper/2310.19721/) (2023)
- [From Monocular Vision to Autonomous Action: Guiding Tumor Resection via 3D Reconstruction](https://mlsanity.com/paper/2503.16263/) (2025)
- [Autonomous Vision-Guided Resection of Central Airway Obstruction](https://mlsanity.com/paper/2502.18586/) (2025)
- [Deep Learning-based Unsupervised Domain Adaptation via a Unified Model for Prostate Lesion Detection Using Multisite Bi-parametric MRI Datasets](https://mlsanity.com/paper/2408.04777/) (2024)

## Built on by (most cited first, 3 of 3)

- [Lesion-centered 3D mapping of colonoscopy procedures: validation of a hierarchical ensemble pipeline on public benchmark videos](https://mlsanity.com/paper/2609.16672/) (2026)
- [Differentiable Mesh State Estimation via Factor Graph Inference for Deformable Object Reconstruction](https://mlsanity.com/paper/2609.16686/) (2026)
- [RAUL: Reference-Assisted Ureteroscopy Localization for Skill Assessment](https://mlsanity.com/paper/2609.19236/) (2026)

## Abstract (arXiv)

Purpose: Natural orifice surgeries minimize the need for incisions and reduce the recovery time compared to open surgery; however, they require a higher level of expertise due to visualization and orientation challenges. We propose a perception pipeline for these surgeries that allows semantic scene understanding. Methods: We bring learning-based segmentation, depth estimation, and 3D reconstruction modules together to create real-time segmented maps of the surgical scenes. Additionally, we use registration with robot poses to solve the scale ambiguity of mapping from monocular images, and allow the use of semantically informed real-time reconstructions in robotic surgeries. Results: We achieve sub-milimeter reconstruction accuracy based on average one-sided Chamfer distances, average pose registration RMSE of 0.9 mm, and an estimated scale within 2% of ground truth. Conclusion: We present a modular perception pipeline, integrating semantic segmentation with real-time monocular SLAM for natural orifice surgeries. This pipeline offers a promising solution for scene understanding that can facilitate automation or surgeon guidance.