# DD-VNB: A Depth-based Dual-Loop Framework for Real-time Visually Navigated Bronchoscopy

arXiv: https://arxiv.org/abs/2403.01683 · HTML: https://mlsanity.com/paper/2403.01683/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 37 references that are in the map.
- First posted on arXiv 4 Mar 2024.

## Builds on (most cited first, 15 of 15)

- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [Unsupervised Learning of Depth and Ego-Motion from Video](https://mlsanity.com/paper/1704.07813/) (2017)
- [Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Video](https://mlsanity.com/paper/1908.10553/) (2019)
- [Understanding the Limitations of CNN-based Absolute Camera Pose Regression](https://mlsanity.com/paper/1903.07504/) (2019)
- [DeepVO: Towards End-to-End Visual Odometry with Deep Recurrent Convolutional Neural Networks](https://mlsanity.com/paper/1709.08429/) (2017)
- [Self-Supervised Monocular Depth and Ego-Motion Estimation in Endoscopy: Appearance Flow to the Rescue](https://mlsanity.com/paper/2112.08122/) (2021)
- [Deep Auxiliary Learning for Visual Localization and Odometry](https://mlsanity.com/paper/1803.03642/) (2018)
- [VINet: Visual-Inertial Odometry as a Sequence-to-Sequence Learning Problem](https://mlsanity.com/paper/1701.08376/) (2017)
- [Augmenting Colonoscopy using Extended and Directional CycleGAN for Lossy Image Translation](https://mlsanity.com/paper/2003.12473/) (2020)
- [Long-Term Temporally Consistent Unpaired Video Translation from Simulated Surgical 3D Data](https://mlsanity.com/paper/2103.17204/) (2021)
- [Adversarial Domain Feature Adaptation for Bronchoscopic Depth Estimation](https://mlsanity.com/paper/2109.11798/) (2021)
- [Autonomous Driving in the Lung using Deep Learning for Localization](https://mlsanity.com/paper/1907.08136/) (2019)
- [OffsetNet: Deep Learning for Localization in the Lung using Rendered Images](https://mlsanity.com/paper/1809.05645/) (2018)
- [BronchoPose: an analysis of data and model configuration for vision-based bronchoscopy pose estimation](https://mlsanity.com/paper/2204.11982/) (2022)

## Built on by (most cited first, 6 of 6)

- [EndoOmni: Zero-Shot Cross-Dataset Depth Estimation in Endoscopy by Robust Self-Learning from Noisy Labels](https://mlsanity.com/paper/2409.05442/) (2024)
- [PANS: Probabilistic Airway Navigation System for Real-time Robust Bronchoscope Localization](https://mlsanity.com/paper/2407.05554/) (2024)
- [BREATH-VL: Vision-Language-Guided 6-DoF Bronchoscopy Localization via Semantic-Geometric Fusion](https://mlsanity.com/paper/2601.03713/) (2026)
- [Harnessing Foundation Models for Robust and Generalizable 6-DOF Bronchoscopy Localization](https://mlsanity.com/paper/2505.24249/) (2025)
- [Enhancing Bronchoscopy Depth Estimation through Synthetic-to-Real Domain Adaptation](https://mlsanity.com/paper/2411.04404/) (2024)
- [BronchoTop: Bronchoscopy Navigation via RGB-Only Topological Localization](https://mlsanity.com/paper/2609.28328/) (2026)

## Abstract (arXiv)

Real-time 6 DOF localization of bronchoscopes is crucial for enhancing intervention quality. However, current vision-based technologies struggle to balance between generalization to unseen data and computational speed. In this study, we propose a Depth-based Dual-Loop framework for real-time Visually Navigated Bronchoscopy (DD-VNB) that can generalize across patient cases without the need of re-training. The DD-VNB framework integrates two key modules: depth estimation and dual-loop localization. To address the domain gap among patients, we propose a knowledge-embedded depth estimation network that maps endoscope frames to depth, ensuring generalization by eliminating patient-specific textures. The network embeds view synthesis knowledge into a cycle adversarial architecture for scale-constrained monocular depth estimation. For real-time performance, our localization module embeds a fast ego-motion estimation network into the loop of depth registration. The ego-motion inference network estimates the pose change of the bronchoscope in high frequency while depth registration against the pre-operative 3D model provides absolute pose periodically. Specifically, the relative pose changes are fed into the registration process as the initial guess to boost its accuracy and speed. Experiments on phantom and in-vivo data from patients demonstrate the effectiveness of our framework: 1) monocular depth estimation outperforms SOTA, 2) localization achieves an accuracy of Absolute Tracking Error (ATE) of 4.7 ± 3.17 mm in phantom and 6.49 ± 3.88 mm in patient data, 3) with a frame-rate approaching video capture speed, 4) without the necessity of case-wise network retraining. The framework's superior speed and accuracy demonstrate its promising clinical potential for real-time bronchoscopic navigation.