# OffsetNet: Deep Learning for Localization in the Lung using Rendered Images

arXiv: https://arxiv.org/abs/1809.05645 · HTML: https://mlsanity.com/paper/1809.05645/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Sep 2018.

## Built on by (most cited first, 9 of 9)

- [BronchoTrack: Airway Lumen Tracking for Branch-Level Bronchoscopic Localization](https://mlsanity.com/paper/2402.12763/) (2024)
- [BronchoPose: an analysis of data and model configuration for vision-based bronchoscopy pose estimation](https://mlsanity.com/paper/2204.11982/) (2022)
- [DD-VNB: A Depth-based Dual-Loop Framework for Real-time Visually Navigated Bronchoscopy](https://mlsanity.com/paper/2403.01683/) (2024)
- [PANS: Probabilistic Airway Navigation System for Real-time Robust Bronchoscope Localization](https://mlsanity.com/paper/2407.05554/) (2024)
- [Airway Label Prediction in Video Bronchoscopy: Capturing Temporal Dependencies Utilizing Anatomical Knowledge](https://mlsanity.com/paper/2307.08318/) (2023)
- [SoK: Security and Privacy Risks of Healthcare AI](https://mlsanity.com/paper/2409.07415/) (2024)
- [BREATH-VL: Vision-Language-Guided 6-DoF Bronchoscopy Localization via Semantic-Geometric Fusion](https://mlsanity.com/paper/2601.03713/) (2026)
- [REMOTE: Real-time Ego-motion Tracking for Various Endoscopes via Multimodal Visual Feature Learning](https://mlsanity.com/paper/2501.18124/) (2025)
- [Airway Segmentation Network for Enhanced Tubular Feature Extraction](https://mlsanity.com/paper/2507.06581/) (2025)

## Abstract (arXiv)

Navigating surgical tools in the dynamic and tortuous anatomy of the lung's airways requires accurate, real-time localization of the tools with respect to the preoperative scan of the anatomy. Such localization can inform human operators or enable closed-loop control by autonomous agents, which would require accuracy not yet reported in the literature. In this paper, we introduce a deep learning architecture, called OffsetNet, to accurately localize a bronchoscope in the lung in real-time. After training on only 30 minutes of recorded camera images in conserved regions of a lung phantom, OffsetNet tracks the bronchoscope's motion on a held-out recording through these same regions at an update rate of 47 Hz and an average position error of 1.4 mm. Because this model performs poorly in less conserved regions, we augment the training dataset with simulated images from these regions. To bridge the gap between camera and simulated domains, we implement domain randomization and a generative adversarial network (GAN). After training on simulated images, OffsetNet tracks the bronchoscope's motion in less conserved regions at an average position error of 2.4 mm, which meets conservative thresholds required for successful tracking.