# 3D endoscopic depth estimation using 3D surface-aware constraints

arXiv: https://arxiv.org/abs/2203.02131 · HTML: https://mlsanity.com/paper/2203.02131/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 27 references that are in the map.
- First posted on arXiv 4 Mar 2022.

## Builds on (most cited first, 15 of 15)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation](https://mlsanity.com/paper/1512.02134/) (2015)
- [Group-wise Correlation Stereo Network](https://mlsanity.com/paper/1903.04025/) (2019)
- [A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promises](https://mlsanity.com/paper/2008.09104/) (2020)
- [Revisiting Stereo Depth Estimation From a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2011.02910/) (2020)
- [Revisiting Single Image Depth Estimation: Toward Higher Resolution Maps with Accurate Object Boundaries](https://mlsanity.com/paper/1803.08673/) (2018)
- [Stereo Correspondence and Reconstruction of Endoscopic Data Challenge](https://mlsanity.com/paper/2101.01133/) (2021)
- [A Survey on Deep Learning Techniques for Stereo-based Depth Estimation](https://mlsanity.com/paper/2006.02535/) (2020)
- [Endo-Depth-and-Motion: Reconstruction and Tracking in Endoscopic Videos using Depth Networks and Photometric Constraints](https://mlsanity.com/paper/2103.16525/) (2021)
- [Normal Assisted Stereo Depth Estimation](https://mlsanity.com/paper/1911.10444/) (2019)
- [E-DSSR: Efficient Dynamic Surgical Scene Reconstruction with Transformer-based Stereoscopic Depth Perception](https://mlsanity.com/paper/2107.00229/) (2021)
- [Adaptive Surface Normal Constraint for Depth Estimation](https://mlsanity.com/paper/2103.15483/) (2021)
- [Self-Supervised Generative Adversarial Network for Depth Estimation in Laparoscopic Images](https://mlsanity.com/paper/2107.04644/) (2021)
- [Self-supervised Learning for Dense Depth Estimation in Monocular Endoscopy](https://mlsanity.com/paper/1806.09521/) (2018)
- [Integrating Artificial Intelligence and Augmented Reality in Robotic Surgery: An Initial dVRK Study Using a Surgical Education Scenario](https://mlsanity.com/paper/2201.00383/) (2022)

## Built on by (most cited first, 3 of 3)

- [Tracking and Mapping in Medical Computer Vision: A Review](https://mlsanity.com/paper/2310.11475/) (2023)
- [Endoscopic Depth Estimation Based on Deep Learning: A Survey](https://mlsanity.com/paper/2507.20881/) (2025)
- [Surg-InvNeRF: Invertible NeRF for 3D tracking and reconstruction in surgical vision](https://mlsanity.com/paper/2508.09681/) (2025)

## Abstract (arXiv)

Robotic-assisted surgery allows surgeons to conduct precise surgical operations with stereo vision and flexible motor control. However, the lack of 3D spatial perception limits situational awareness during procedures and hinders mastering surgical skills in the narrow abdominal space. Depth estimation, as a representative perception task, is typically defined as an image reconstruction problem. In this work, we show that depth estimation can be reformed from a 3D surface perspective. We propose a loss function for depth estimation that integrates the surface-aware constraints, leading to a faster and better convergence with the valid information from spatial information. In addition, camera parameters are incorporated into the training pipeline to increase the control and transparency of the depth estimation. We also integrate a specularity removal module to recover more buried image information. Quantitative experimental results on endoscopic datasets and user studies with medical professionals demonstrate the effectiveness of our method.