# Bimodal Camera Pose Prediction for Endoscopy arXiv: https://arxiv.org/abs/2204.04968 ยท HTML: https://mlsanity.com/paper/2204.04968/ ## Key facts - Cited by 20 papers in the map (data through 14 Oct 2026). - Builds on 19 of its 43 references that are in the map. - First posted on arXiv 11 Apr 2022. ## Builds on (most cited first, 19 of 19) - [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015) - [Unsupervised Learning of Depth and Ego-Motion from Video](https://mlsanity.com/paper/1704.07813/) (2017) - [Geometric Loss Functions for Camera Pose Regression with Deep Learning](https://mlsanity.com/paper/1704.00390/) (2017) - [BA-Net: Dense Bundle Adjustment Network](https://mlsanity.com/paper/1806.04807/) (2018) - [Neighbourhood Consensus Networks](https://mlsanity.com/paper/1810.10510/) (2018) - [Geometry-Aware Learning of Maps for Camera Localization](https://mlsanity.com/paper/1712.03342/) (2017) - [Unsupervised Reverse Domain Adaptation for Synthetic Medical Images via Adversarial Training](https://mlsanity.com/paper/1711.06606/) (2017) - [Visual Camera Re-Localization Using Graph Neural Networks and Relative Pose Supervision](https://mlsanity.com/paper/2104.02538/) (2021) - [Augmenting Colonoscopy using Extended and Directional CycleGAN for Lossy Image Translation](https://mlsanity.com/paper/2003.12473/) (2020) - [A Mixed Classification-Regression Framework for 3D Pose Estimation from 2D Images](https://mlsanity.com/paper/1805.03225/) (2018) - [Unsupervised Odometry and Depth Learning for Endoscopic Capsule Robots](https://mlsanity.com/paper/1803.01047/) (2018) - [Detecting Deficient Coverage in Colonoscopies](https://mlsanity.com/paper/2001.08589/) (2020) - [SLAM based Quasi Dense Reconstruction For Minimally Invasive Surgery Scenes](https://mlsanity.com/paper/1705.09107/) (2017) - [Deep EndoVO: A Recurrent Convolutional Neural Network (RCNN) based Visual Odometry Approach for Endoscopic Capsule Robots](https://mlsanity.com/paper/1708.06822/) (2017) - [Motion-based Camera Localization System in Colonoscopy Videos](https://mlsanity.com/paper/2012.01690/) (2020) - [Learning monocular visual odometry with dense 3D mapping from dense 3D flow](https://mlsanity.com/paper/1803.02286/) (2018) - [SuperPoint features in endoscopy](https://mlsanity.com/paper/2203.04302/) (2022) - [Learning-Based Depth and Pose Estimation for Monocular Endoscope with Loss Generalization](https://mlsanity.com/paper/2107.13263/) (2021) - [Improving the generalization of network based relative pose regression: dimension reduction as a regularizer](https://mlsanity.com/paper/2010.12796/) (2020) ## Built on by (most cited first, 20 of 20) - [Colonoscopy 3D Video Dataset with Paired Depth from 2D-3D Registration](https://mlsanity.com/paper/2206.08903/) (2022) - [EndoMapper dataset of complete calibrated endoscopy procedures](https://mlsanity.com/paper/2204.14240/) (2022) - [Tracking and Mapping in Medical Computer Vision: A Review](https://mlsanity.com/paper/2310.11475/) (2023) - [EndoGSLAM: Real-Time Dense Reconstruction and Tracking in Endoscopic Surgeries using Gaussian Splatting](https://mlsanity.com/paper/2403.15124/) (2024) - [SimCol3D -- 3D Reconstruction during Colonoscopy Challenge](https://mlsanity.com/paper/2307.11261/) (2023) - [LightDepth: Single-View Depth Self-Supervision from Illumination Decline](https://mlsanity.com/paper/2308.10525/) (2023) - [C3VDv2 -- Colonoscopy 3D video dataset with enhanced realism](https://mlsanity.com/paper/2506.24074/) (2025) - [Photometric single-view dense 3D reconstruction in endoscopy](https://mlsanity.com/paper/2204.09083/) (2022) - [Advancing Dense Endoscopic Reconstruction with Gaussian Splatting-driven Surface Normal-aware Tracking and Mapping](https://mlsanity.com/paper/2501.19319/) (2025) - [MonoPCC: Photometric-invariant Cycle Constraint for Monocular Depth Estimation of Endoscopic Images](https://mlsanity.com/paper/2404.16571/) (2024) - [Task-guided Domain Gap Reduction for Monocular Depth Prediction in Endoscopy](https://mlsanity.com/paper/2310.01663/) (2023) - [ColonNeRF: High-Fidelity Neural Reconstruction of Long Colonoscopy](https://mlsanity.com/paper/2312.02015/) (2023) - [NeRFscopy: Neural Radiance Fields for in-vivo Time-Varying Tissues from Endoscopy](https://mlsanity.com/paper/2602.15775/) (2026) - [EndoPBR: Material and Lighting Estimation for Photorealistic Surgical Simulations via Physically-based Rendering](https://mlsanity.com/paper/2502.20669/) (2025) - [Learning to Efficiently Adapt Foundation Models for Self-Supervised Endoscopic 3D Scene Reconstruction from Any Cameras](https://mlsanity.com/paper/2503.15917/) (2025) - [TR2M: Transferring Monocular Relative Depth to Metric Depth with Language Descriptions and Dual-Level Scale-Oriented Contrast](https://mlsanity.com/paper/2506.13387/) (2025) - [Uncertainty and Self-Supervision in Single-View Depth](https://mlsanity.com/paper/2406.14226/) (2024) - [REMOTE: Real-time Ego-motion Tracking for Various Endoscopes via Multimodal Visual Feature Learning](https://mlsanity.com/paper/2501.18124/) (2025) - [Surg-InvNeRF: Invertible NeRF for 3D tracking and reconstruction in surgical vision](https://mlsanity.com/paper/2508.09681/) (2025) - [Structure-to-Image: Zero-Shot Depth Estimation in Colonoscopy via High-Fidelity Sim-to-Real Adaptation](https://mlsanity.com/paper/2602.21740/) (2026) ## Abstract (arXiv) Deducing the 3D structure of endoscopic scenes from images is exceedingly challenging. In addition to deformation and view-dependent lighting, tubular structures like the colon present problems stemming from their self-occluding and repetitive anatomical structure. In this paper, we propose SimCol, a synthetic dataset for camera pose estimation in colonoscopy, and a novel method that explicitly learns a bimodal distribution to predict the endoscope pose. Our dataset replicates real colonoscope motion and highlights the drawbacks of existing methods. We publish 18k RGB images from simulated colonoscopy with corresponding depth and camera poses and make our data generation environment in Unity publicly available. We evaluate different camera pose prediction methods and demonstrate that, when trained on our data, they generalize to real colonoscopy sequences, and our bimodal approach outperforms prior unimodal work.