Geometry-Consistent Endoscopic Representations for Image-Guided Navigation via Structured Foundation Model Adaptation
Not on the map. Why?
Abstract (arXiv)
Accurate vision-based navigation in monocular endoscopy is difficult due to limited depth cues, weak tissue texture, non-rigid deformation, and substantial appearance variation across domains, all of which complicate pose estimation, depth prediction, and image-to-anatomy alignment. Although recent vision foundation models have shown promise, their learned representations often remain insufficiently geometry-consistent, hindering stable feature correspondence and limiting their reliability for downstream navigation tasks. We propose a unified framework for learning geometry-consistent and domain-robust image representations for monocular endoscopy. The framework combines a synthetic data pipeline that provides accurate geometric supervision with Hierarchy-Aware Geometry-Semantic Adaptation, a structured alternative to standard LoRA that inserts low-rank adapters selectively across the transformer hierarchy and couples them with layer-wise training objectives to encourage geometric correspondence in intermediate features and semantic consistency in deeper features. Experiments on public and proprietary datasets show improved geometric and semantic representation quality, leading to better performance on downstream navigation tasks including pose estimation and monocular depth estimation. The learned representations show favorable synthetic-to-real transfer on clinical bronchoscopy and provide a useful initialization for adaptation to sinus endoscopy and colonoscopy under limited supervision. The framework also shows favorable scaling with model size and training data. These results support hierarchy-aware, geometry-guided adaptation as a practical approach for endoscopic representation learning.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 28 of its 38 references that are in the map. First posted on arXiv 15 Jun 2026.
Builds on
28 of 38 references.
References · 28
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
- 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
- 2021Vision Transformers for Dense Predictioncited by 1,789
- 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
- 2018World Modelscited by 1,444
- 2024Depth Anything V2cited by 1,385
- 2025DINOv3cited by 1,297
- 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,264
16 more
- 2020Contrastive Learning for Unpaired Image-to-Image Translationcited by 637
- 2023AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuningcited by 337
- 2022ST-Adapter: Parameter-Efficient Image-to-Video Transfer Learningcited by 192
- 2024Medical SAM 2: Segment medical images as video via Segment Anything Model 2cited by 151
- 2023SurgicalSAM: Efficient Class Promptable Surgical Instrument Segmentationcited by 103
- 2022Colonoscopy 3D Video Dataset with Paired Depth from 2D-3D Registrationcited by 78
- 2023Foundation Model for Endoscopy Video Analysis via Large-scale Self-supervised Pre-traincited by 75
- 2024Surgical-DINO: Adapter Learning of Foundation Models for Depth Estimation in Endoscopic Surgerycited by 52
- 2025What Is The Best 3D Scene Representation for Robotics? From Geometric to Foundation Modelscited by 29
- 2024FastSAM3D: An Efficient Segment Anything Model for 3D Volumetric Medical Imagescited by 14
- 2024Leveraging Near-Field Lighting for Monocular Depth Estimation from Endoscopy Videoscited by 14
- 2025EndoMamba: An Efficient Foundation Model for Endoscopic Videos via Hierarchical Pre-trainingcited by 11
- 2025EndoDINO: A Foundation Model for GI Endoscopycited by 9
- 2024FluoroSAM: A Language-promptable Foundation Model for Flexible X-ray Image Segmentationcited by 7
- 2025Adaptation of Foundation Models for Medical Image Analysis: Strategies, Challenges, and Future Directionscited by 3
- 2025Monocular absolute depth estimation from endoscopy via domain-invariant feature learning and latent consistencycited by 2
Built on by
Not cited yet.