# Combining Deep Learning with Geometric Features for Image based Localization in the Gastrointestinal Tract

arXiv: https://arxiv.org/abs/2005.05481 · HTML: https://mlsanity.com/paper/2005.05481/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 May 2020.

## Built on by (most cited first, 3 of 3)

- [Bayesian dense inverse searching algorithm for real-time stereo matching in minimally invasive surgery](https://mlsanity.com/paper/2106.07136/) (2021)
- [SLAM assisted 3D tracking system for laparoscopic surgery](https://mlsanity.com/paper/2409.11688/) (2024)
- [LapaTrack-3D: 6 DoF pre-operative shape tracking for laparoscopic surgery](https://mlsanity.com/paper/2609.19954/) (2026)

## Abstract (arXiv)

Tracking monocular colonoscope in the Gastrointestinal tract (GI) is a challenging problem as the images suffer from deformation, blurred textures, significant changes in appearance. They greatly restrict the tracking ability of conventional geometry based methods. Even though Deep Learning (DL) can overcome these issues, limited labeling data is a roadblock to state-of-art DL method. Considering these, we propose a novel approach to combine DL method with traditional feature based approach to achieve better localization with small training data. Our method fully exploits the best of both worlds by introducing a Siamese network structure to perform few-shot classification to the closest zone in the segmented training image set. The classified label is further adopted to initialize the pose of scope. To fully use the training dataset, a pre-generated triangulated map points within the zone in the training set are registered with observation and contribute to estimating the optimal pose of the test image. The proposed hybrid method is extensively tested and compared with existing methods, and the result shows significant improvement over traditional geometric based or DL based localization. The accuracy is improved by 28.94% (Position) and 10.97% (Orientation) with respect to state-of-art method.