# Deep Learning and Conditional Random Fields-based Depth Estimation and Topographical Reconstruction from Conventional Endoscopy

arXiv: https://arxiv.org/abs/1710.11216 · HTML: https://mlsanity.com/paper/1710.11216/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Oct 2017.

## Built on by (most cited first, 30 of 31)

- [Colonoscopy 3D Video Dataset with Paired Depth from 2D-3D Registration](https://mlsanity.com/paper/2206.08903/) (2022)
- [Self-Supervised Monocular Depth and Ego-Motion Estimation in Endoscopy: Appearance Flow to the Rescue](https://mlsanity.com/paper/2112.08122/) (2021)
- [SimCol3D -- 3D Reconstruction during Colonoscopy Challenge](https://mlsanity.com/paper/2307.11261/) (2023)
- [LightDepth: Single-View Depth Self-Supervision from Illumination Decline](https://mlsanity.com/paper/2308.10525/) (2023)
- [C3VDv2 -- Colonoscopy 3D video dataset with enhanced realism](https://mlsanity.com/paper/2506.24074/) (2025)
- [Depth Anything in Medical Images: A Comparative Study](https://mlsanity.com/paper/2401.16600/) (2024)
- [PanoDepth: A Two-Stage Approach for Monocular Omnidirectional Depth Estimation](https://mlsanity.com/paper/2202.01323/) (2022)
- [A geometry-aware deep network for depth estimation in monocular endoscopy](https://mlsanity.com/paper/2304.10241/) (2023)
- [Multi-task learning with cross-task consistency for improved depth estimation in colonoscopy](https://mlsanity.com/paper/2311.18664/) (2023)
- [On the Uncertain Single-View Depths in Colonoscopies](https://mlsanity.com/paper/2112.08906/) (2021)
- [Bidirectional Semi-supervised Dual-branch CNN for Robust 3D Reconstruction of Stereo Endoscopic Images via Adaptive Cross and Parallel Supervisions](https://mlsanity.com/paper/2210.08291/) (2022)
- [Structure-preserving Image Translation for Depth Estimation in Colonoscopy Video](https://mlsanity.com/paper/2408.10153/) (2024)
- [SyntheX: Scaling Up Learning-based X-ray Image Analysis Through In Silico Experiments](https://mlsanity.com/paper/2206.06127/) (2022)
- [Task-guided Domain Gap Reduction for Monocular Depth Prediction in Endoscopy](https://mlsanity.com/paper/2310.01663/) (2023)
- [Learning-Based Depth and Pose Estimation for Monocular Endoscope with Loss Generalization](https://mlsanity.com/paper/2107.13263/) (2021)
- [A comprehensive survey on recent deep learning-based methods applied to surgical data](https://mlsanity.com/paper/2209.01435/) (2022)
- [ColonCrafter: A Depth Estimation Model for Colonoscopy Videos Using Diffusion Priors](https://mlsanity.com/paper/2509.13525/) (2025)
- [Leveraging a realistic synthetic database to learn Shape-from-Shading for estimating the colon depth in colonoscopy images](https://mlsanity.com/paper/2311.05021/) (2023)
- [RealSynCol: a high-fidelity synthetic colon dataset for 3D reconstruction applications](https://mlsanity.com/paper/2602.08397/) (2026)
- [Depth Estimation from Single-shot Monocular Endoscope Image Using Image Domain Adaptation And Edge-Aware Depth Estimation](https://mlsanity.com/paper/2201.04485/) (2022)
- [Generalizing monocular colonoscopy image depth estimation by uncertainty-based global and local fusion network](https://mlsanity.com/paper/2409.15006/) (2024)
- [Learning to Efficiently Adapt Foundation Models for Self-Supervised Endoscopic 3D Scene Reconstruction from Any Cameras](https://mlsanity.com/paper/2503.15917/) (2025)
- [Endoscopic Depth Estimation Based on Deep Learning: A Survey](https://mlsanity.com/paper/2507.20881/) (2025)
- [Stereo Video Reconstruction Without Explicit Depth Maps for Endoscopic Surgery](https://mlsanity.com/paper/2109.08227/) (2021)
- [Real-time Virtual Intraoperative CT for Image Guided Surgery](https://mlsanity.com/paper/2112.02608/) (2021)
- [SoftEnNet: Symbiotic Monocular Depth Estimation and Lumen Segmentation for Colonoscopy Endorobots](https://mlsanity.com/paper/2301.08157/) (2023)
- [Self-supervised Monocular Depth and Pose Estimation for Endoscopy with Latent Priors](https://mlsanity.com/paper/2411.17790/) (2024)
- [V2-SfMLearner: Learning Monocular Depth and Ego-motion for Multimodal Wireless Capsule Endoscopy](https://mlsanity.com/paper/2412.17595/) (2024)
- [Multi-Modal Monocular Endoscopic Depth and Pose Estimation with Edge-Guided Self-Supervision](https://mlsanity.com/paper/2602.17785/) (2026)
- [Structure-to-Image: Zero-Shot Depth Estimation in Colonoscopy via High-Fidelity Sim-to-Real Adaptation](https://mlsanity.com/paper/2602.21740/) (2026)

## Abstract (arXiv)

Colorectal cancer is the fourth leading cause of cancer deaths worldwide and the second leading cause in the United States. The risk of colorectal cancer can be mitigated by the identification and removal of premalignant lesions through optical colonoscopy. Unfortunately, conventional colonoscopy misses more than 20% of the polyps that should be removed, due in part to poor contrast of lesion topography. Imaging tissue topography during a colonoscopy is difficult because of the size constraints of the endoscope and the deforming mucosa. Most existing methods make geometric assumptions or incorporate a priori information, which limits accuracy and sensitivity. In this paper, we present a method that avoids these restrictions, using a joint deep convolutional neural network-conditional random field (CNN-CRF) framework. Estimated depth is used to reconstruct the topography of the surface of the colon from a single image. We train the unary and pairwise potential functions of a CRF in a CNN on synthetic data, generated by developing an endoscope camera model and rendering over 100,000 images of an anatomically-realistic colon. We validate our approach with real endoscopy images from a porcine colon, transferred to a synthetic-like domain, with ground truth from registered computed tomography measurements. The CNN-CRF approach estimates depths with a relative error of 0.152 for synthetic endoscopy images and 0.242 for real endoscopy images. We show that the estimated depth maps can be used for reconstructing the topography of the mucosa from conventional colonoscopy images. This approach can easily be integrated into existing endoscopy systems and provides a foundation for improving computer-aided detection algorithms for detection, segmentation and classification of lesions.