mlsanityData to 14 Oct 2026

Self-supervised Monocular Depth and Pose Estimation for Endoscopy with Latent Priors

arXiv 2411.17790 ↗First posted 26 Nov 2024cs.CV cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Accurate 3D mapping in endoscopy enables quantitative, holistic lesion characterization within the gastrointestinal (GI) tract, requiring reliable depth and pose estimation. However, endoscopy systems are monocular, and existing methods relying on synthetic datasets or complex models often lack generalizability in challenging endoscopic conditions. We propose a robust self-supervised monocular depth and pose estimation framework that incorporates a Generative Latent Bank and a Variational Autoencoder (VAE). The Generative Latent Bank leverages extensive depth scenes from natural images to condition the depth network, enhancing realism and robustness of depth predictions through latent feature priors. For pose estimation, we reformulate it within a VAE framework, treating pose transitions as latent variables to regularize scale, stabilize z-axis prominence, and improve x-y sensitivity. This dual refinement pipeline enables accurate depth and pose predictions, effectively addressing the GI tract's complex textures and lighting. Extensive evaluations on SimCol and EndoSLAM datasets confirm our framework's superior performance over published self-supervised methods in endoscopic depth and pose estimation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 36 of its 58 references that are in the map. First posted on arXiv 26 Nov 2024.

Builds on

36 of 58 references.

References · 36

  1. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  2. 2017Improved Training of Wasserstein GANscited by 2,235
  3. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 2,010
  4. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,360
  5. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,274
  6. 2016Unsupervised Monocular Depth Estimation with Left-Right Consistencycited by 820
  7. 2018Deep Ordinal Regression Network for Monocular Depth Estimationcited by 723
  8. 2021DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Camerascited by 687
  9. 2014Semi-Supervised Learning with Deep Generative Modelscited by 496
  10. 2014Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecturecited by 483
  11. 2024Depth Pro: Sharp Monocular Metric Depth in Less Than a Secondcited by 404
  12. 2016Deeper Depth Prediction with Fully Convolutional Residual Networkscited by 403
24 more
  1. 2019From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimationcited by 382
  2. 2020D3VO: Deep Depth, Deep Pose and Deep Uncertainty for Monocular Visual Odometrycited by 206
  3. 2020On the uncertainty of self-supervised monocular depth estimationcited by 182
  4. 2015Learning Depth from Single Monocular Images Using Deep Convolutional Neural Fieldscited by 169
  5. 2020GLEAN: Generative Latent Bank for Large-Factor Image Super-Resolutioncited by 158
  6. 2017Learning Depth from Monocular Videos using Direct Methodscited by 154
  7. 2022MonoViT: Self-Supervised Monocular Depth Estimation with a Vision Transformercited by 136
  8. 2022Lite-Mono: A Lightweight CNN and Transformer Architecture for Self-Supervised Monocular Depth Estimationcited by 100
  9. 2022Colonoscopy 3D Video Dataset with Paired Depth from 2D-3D Registrationcited by 80
  10. 2021Fine-grained Semantics-aware Representation Enhancement for Self-supervised Monocular Depth Estimationcited by 55
  11. 2019Unsupervised Monocular Depth and Ego-motion Learning with Structure and Semanticscited by 49
  12. 2020SynDistNet: Self-Supervised Monocular Fisheye Camera Distance Estimation Synergized with Semantic Segmentation for Autonomous Drivingcited by 48
  13. 2018LEGO: Learning Edge with Geometry all at Once by Watching Videoscited by 47
  14. 2024EndoDAC: Efficient Adapting Foundation Model for Self-Supervised Depth Estimation from Any Endoscopic Cameracited by 43
  15. 2017Unsupervised Learning of Geometry with Edge-aware Depth-Normal Consistencycited by 40
  16. 2018Deep attention-based classification network for robust depth predictioncited by 36
  17. 2016SceneNet RGB-D: 5M Photorealistic Images of Synthetic Indoor Trajectories with Ground Truthcited by 32
  18. 2017Deep Learning and Conditional Random Fields-based Depth Estimation and Topographical Reconstruction from Conventional Endoscopycited by 31
  19. 2023SimCol3D -- 3D Reconstruction during Colonoscopy Challengecited by 19
  20. 2023LightDepth: Single-View Depth Self-Supervision from Illumination Declinecited by 17
  21. 2023DualRefine: Self-Supervised Depth and Pose Estimation Through Iterative Epipolar Sampling and Refinement Toward Equilibriumcited by 16
  22. 2023Multi-task learning with cross-task consistency for improved depth estimation in colonoscopycited by 8
  23. 2023MonoProb: Self-Supervised Monocular Depth Estimation with Interpretable Uncertaintycited by 6
  24. 2021Learning-Based Depth and Pose Estimation for Monocular Endoscope with Loss Generalizationcited by 4

Built on by

Not cited yet.