mlsanityData to 14 Oct 2026

ColDE: A Depth Estimation Framework for Colonoscopy Reconstruction

arXiv 2111.10371 ↗First posted 19 Nov 2021eess.IV cs.CV cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

One of the key elements of reconstructing a 3D mesh from a monocular video is generating every frame's depth map. However, in the application of colonoscopy video reconstruction, producing good-quality depth estimation is challenging. Neural networks can be easily fooled by photometric distractions or fail to capture the complex shape of the colon surface, predicting defective shapes that result in broken meshes. Aiming to fundamentally improve the depth estimation quality for colonoscopy 3D reconstruction, in this work we have designed a set of training losses to deal with the special challenges of colonoscopy data. For better training, a set of geometric consistency objectives was developed, using both depth and surface normal information. Also, the classic photometric loss was extended with feature matching to compensate for illumination noise. With the training losses powerful enough, our self-supervised framework named ColDE is able to produce better depth maps of colonoscopy data as compared to the previous work utilizing prior depth knowledge. Used in reconstruction, our network is able to reconstruct good-quality colon meshes in real-time without any post-processing, making it the first to be clinically applicable.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 35 of its 52 references that are in the map. First posted on arXiv 19 Nov 2021.

Builds on

35 of 52 references.

References · 35

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  3. 2016Unsupervised Monocular Depth Estimation with Left-Right Consistencycited by 820
  4. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 782
  5. 2018MegaDepth: Learning Single-View Depth Prediction from Internet Photoscited by 737
  6. 2018Deep Ordinal Regression Network for Monocular Depth Estimationcited by 723
  7. 2016Direct Sparse Odometrycited by 485
  8. 2014Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecturecited by 483
  9. 20193D Packing for Self-Supervised Monocular Depth Estimationcited by 439
  10. 2018GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Posecited by 342
  11. 2019Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Videocited by 244
  12. 2018Competitive Collaboration: Joint Unsupervised Learning of Depth, Camera Motion, Optical Flow and Motion Segmentationcited by 222
23 more
  1. 2018Unsupervised Learning of Depth and Ego-Motion from Monocular Video Using 3D Geometric Constraintscited by 179
  2. 2018Unsupervised Learning of Monocular Depth Estimation and Visual Odometry with Deep Feature Reconstructioncited by 159
  3. 2018Depth Prediction Without the Sensors: Leveraging Structure for Unsupervised Learning from Monocular Videoscited by 155
  4. 2017Learning Depth from Monocular Videos using Direct Methodscited by 154
  5. 2018DF-Net: Unsupervised Joint Learning of Depth and Flow using Cross-Task Consistencycited by 148
  6. 2020Semantically-Guided Representation Learning for Self-Supervised Monocular Depthcited by 134
  7. 2020HR-Depth: High Resolution Self-Supervised Monocular Depth Estimationcited by 115
  8. 2017Semi-Supervised Deep Learning for Monocular Depth Map Predictioncited by 109
  9. 2017SfM-Net: Learning of Structure and Motion from Videocited by 89
  10. 2018Deep Virtual Stereo Odometry: Leveraging Deep Depth Prediction for Monocular Direct Sparse Odometrycited by 85
  11. 2020Unsupervised Monocular Depth Learning in Dynamic Scenescited by 84
  12. 2017Unsupervised Reverse Domain Adaptation for Synthetic Medical Images via Adversarial Trainingcited by 57
  13. 2019Recurrent Neural Network for (Un-)supervised Learning of Monocular VideoVisual Odometry and Depthcited by 50
  14. 2018LEGO: Learning Edge with Geometry all at Once by Watching Videoscited by 47
  15. 2020Learning Monocular Visual Odometry via Self-Supervised Long-Term Modelingcited by 39
  16. 2021MonoIndoor: Towards Good Practice of Self-Supervised Monocular Depth Estimation for Indoor Environmentscited by 38
  17. 2021StructDepth: Leveraging the structural regularities for self-supervised indoor depth estimationcited by 36
  18. 2020Augmenting Colonoscopy using Extended and Directional CycleGAN for Lossy Image Translationcited by 29
  19. 2020Reconstructing Sinus Anatomy from Endoscopic Video -- Towards a Radiation-free Approach for Quantitative Longitudinal Assessmentcited by 23
  20. 2018SIGNet: Semantic Instance Aided Unsupervised 3D Geometry Perceptioncited by 16
  21. 2021Towards Interpretable Deep Networks for Monocular Depth Estimationcited by 12
  22. 2019Self-supervised Learning for Single View Depth and Surface Normal Estimationcited by 8
  23. 2021Lighting Enhancement Aids Reconstruction of Colonoscopic Surfacescited by 8

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 2; 2024: 3; 2025: 1.

7 papers.

  1. 2023SENDD: Sparse Efficient Neural Depth and Deformation for Tissue Trackingcited by 14
  2. 2024Leveraging Near-Field Lighting for Monocular Depth Estimation from Endoscopy Videoscited by 14
  3. 2023Multi-task learning with cross-task consistency for improved depth estimation in colonoscopycited by 8
  4. 2022C3Fusion: Consistent Contrastive Colon Fusion, Towards Deep SLAM in Colonoscopycited by 7
  5. 2024NFL-BA: Near-Field Light Bundle Adjustment for SLAM in Dynamic Lightingcited by 4
  6. 2025Combinative Matching for Geometric Shape Assemblycited by 3
  7. 2024ToDER: Towards Colonoscopy Depth Estimation and Reconstruction with Geometry Constraint Adaptationcited by 2