# Endo-FASt3r: Endoscopic Foundation model Adaptation for Structure from motion

arXiv: https://arxiv.org/abs/2503.07204 · HTML: https://mlsanity.com/paper/2503.07204/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 33 references that are in the map.
- First posted on arXiv 10 Mar 2025.

## Builds on (most cited first, 21 of 21)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Depth Anything V2](https://mlsanity.com/paper/2406.09414/) (2024)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [Unsupervised Learning of Depth and Ego-Motion from Video](https://mlsanity.com/paper/1704.07813/) (2017)
- [DoRA: Weight-Decomposed Low-Rank Adaptation](https://mlsanity.com/paper/2402.09353/) (2024)
- [Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks](https://mlsanity.com/paper/1602.07868/) (2016)
- [Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Video](https://mlsanity.com/paper/1908.10553/) (2019)
- [Learning Depth from Monocular Videos using Direct Methods](https://mlsanity.com/paper/1712.00175/) (2017)
- [Stereo Correspondence and Reconstruction of Endoscopic Data Challenge](https://mlsanity.com/paper/2101.01133/) (2021)
- [Reloc3r: Large-Scale Training of Relative Camera Pose Regression for Generalizable, Fast, and Accurate Visual Localization](https://mlsanity.com/paper/2412.08376/) (2024)
- [Endo-Depth-and-Motion: Reconstruction and Tracking in Endoscopic Videos using Depth Networks and Photometric Constraints](https://mlsanity.com/paper/2103.16525/) (2021)
- [Self-Supervised Monocular Depth and Ego-Motion Estimation in Endoscopy: Appearance Flow to the Rescue](https://mlsanity.com/paper/2112.08122/) (2021)
- [DeFeat-Net: General Monocular Depth via Simultaneous Unsupervised Representation Learning](https://mlsanity.com/paper/2003.13446/) (2020)
- [Depth Any Video with Scalable Synthetic Data](https://mlsanity.com/paper/2410.10815/) (2024)
- [Surgical-DINO: Adapter Learning of Foundation Models for Depth Estimation in Endoscopic Surgery](https://mlsanity.com/paper/2401.06013/) (2024)
- [Learning How To Robustly Estimate Camera Pose in Endoscopic Videos](https://mlsanity.com/paper/2304.08023/) (2023)
- [EndoDAC: Efficient Adapting Foundation Model for Self-Supervised Depth Estimation from Any Endoscopic Camera](https://mlsanity.com/paper/2405.08672/) (2024)
- [Tracking and Mapping in Medical Computer Vision: A Review](https://mlsanity.com/paper/2310.11475/) (2023)
- [MoRA: High-Rank Updating for Parameter-Efficient Fine-Tuning](https://mlsanity.com/paper/2405.12130/) (2024)
- [DARES: Depth Anything in Robotic Endoscopic Surgery with Self-supervised Vector-LoRA of the Foundation Model](https://mlsanity.com/paper/2408.17433/) (2024)

## Built on by (most cited first, 6 of 6)

- [BREATH-VL: Vision-Language-Guided 6-DoF Bronchoscopy Localization via Semantic-Geometric Fusion](https://mlsanity.com/paper/2601.03713/) (2026)
- [EndoSfM3D: Learning to 3D Reconstruct Any Endoscopic Surgery Scene using Self-supervised Foundation Model](https://mlsanity.com/paper/2510.22359/) (2025)
- [3D Consistency Optimization for Self-Supervised Monocular Video Depth Estimation](https://mlsanity.com/paper/2606.15681/) (2026)
- [Geometry-Aware Camera Localization for Bronchoscopy](https://mlsanity.com/paper/2608.07116/) (2026)
- [MV-dVRK: A Multi-Viewpoint Benchmark for Spatial Surgical Perception](https://mlsanity.com/paper/2609.02717/) (2026)
- [SCOPE-4D: Endoscopic 4D Geometry Foundation Models](https://mlsanity.com/paper/2610.02343/) (2026)

## Abstract (arXiv)

Accurate depth and camera pose estimation is essential for achieving high-quality 3D visualisations in robotic-assisted surgery. Despite recent advancements in foundation model adaptation to monocular depth estimation of endoscopic scenes via self-supervised learning (SSL), no prior work has explored their use for pose estimation. These methods rely on low rank-based adaptation approaches, which constrain model updates to a low-rank space. We propose Endo-FASt3r, the first monocular SSL depth and pose estimation framework that uses foundation models for both tasks. We extend the Reloc3r relative pose estimation foundation model by designing Reloc3rX, introducing modifications necessary for convergence in SSL. We also present DoMoRA, a novel adaptation technique that enables higher-rank updates and faster convergence. Experiments on the SCARED dataset show that Endo-FASt3r achieves a substantial 10% improvement in pose estimation and a 2% improvement in depth estimation over prior work. Similar performance gains on the Hamlyn and StereoMIS datasets reinforce the generalisability of Endo-FASt3r across different datasets.