# Self-Supervised Generative Adversarial Network for Depth Estimation in Laparoscopic Images

arXiv: https://arxiv.org/abs/2107.04644 · HTML: https://mlsanity.com/paper/2107.04644/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 37 references that are in the map.
- First posted on arXiv 9 Jul 2021.

## Builds on (most cited first, 20 of 20)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)](https://mlsanity.com/paper/1511.07289/) (2015)
- [Unsupervised Monocular Depth Estimation with Left-Right Consistency](https://mlsanity.com/paper/1609.03677/) (2016)
- [Unsupervised Learning of Depth and Ego-Motion from Video](https://mlsanity.com/paper/1704.07813/) (2017)
- [Pyramid Stereo Matching Network](https://mlsanity.com/paper/1803.08669/) (2018)
- [Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue](https://mlsanity.com/paper/1603.04992/) (2016)
- [DualGAN: Unsupervised Dual Learning for Image-to-Image Translation](https://mlsanity.com/paper/1704.02510/) (2017)
- [Self-Supervised Monocular Depth Hints](https://mlsanity.com/paper/1909.09051/) (2019)
- [Self-supervised Monocular Trained Depth Estimation using Self-attention and Discrete Disparity Volume](https://mlsanity.com/paper/2003.13951/) (2020)
- [DeepPruner: Learning Efficient Stereo Matching via Differentiable PatchMatch](https://mlsanity.com/paper/1909.05845/) (2019)
- [Stereo Correspondence and Reconstruction of Endoscopic Data Challenge](https://mlsanity.com/paper/2101.01133/) (2021)
- [Multiple Meta-model Quantifying for Medical Visual Question Answering](https://mlsanity.com/paper/2105.08913/) (2021)
- [Dense Depth Estimation in Monocular Endoscopy with Self-supervised Learning Methods](https://mlsanity.com/paper/1902.07766/) (2019)
- [Unsupervised Adversarial Depth Estimation using Cycled Generative Networks](https://mlsanity.com/paper/1807.10915/) (2018)
- [Self-Supervised Siamese Learning on Stereo Image Pairs for Depth Estimation in Robotic Surgery](https://mlsanity.com/paper/1705.08260/) (2017)
- [End-to-End Real-time Catheter Segmentation with Optical Flow-Guided Warping during Endovascular Intervention](https://mlsanity.com/paper/2006.09117/) (2020)
- [H-Net: Unsupervised Attention-based Stereo Depth Estimation Leveraging Epipolar Geometry](https://mlsanity.com/paper/2104.11288/) (2021)

## Built on by (most cited first, 21 of 21)

- [Self-Supervised Depth Estimation in Laparoscopic Image using 3D Geometric Consistency](https://mlsanity.com/paper/2208.08407/) (2022)
- [LightDepth: Single-View Depth Self-Supervision from Illumination Decline](https://mlsanity.com/paper/2308.10525/) (2023)
- [SurgicalGS: Dynamic 3D Gaussian Splatting for Accurate Robotic-Assisted Surgical Scene Reconstruction](https://mlsanity.com/paper/2410.09292/) (2024)
- [Light-weight Deformable Registration using Adversarial Learning with Distilling Knowledge](https://mlsanity.com/paper/2110.01293/) (2021)
- [Stereo Dense Scene Reconstruction and Accurate Localization for Learning-Based Navigation of Laparoscope in Minimally Invasive Surgery](https://mlsanity.com/paper/2110.03912/) (2021)
- [Deep Laparoscopic Stereo Matching with Transformers](https://mlsanity.com/paper/2207.12152/) (2022)
- [WS-SfMLearner: Self-supervised Monocular Depth and Ego-motion Estimation on Surgical Videos with Unknown Camera Parameters](https://mlsanity.com/paper/2308.11776/) (2023)
- [SAMSNeRF: Segment Anything Model (SAM) Guides Dynamic Surgical Scene Reconstruction by Neural Radiance Field (NeRF)](https://mlsanity.com/paper/2308.11774/) (2023)
- [On the Uncertain Single-View Depths in Colonoscopies](https://mlsanity.com/paper/2112.08906/) (2021)
- [Adversarial Attacks on Monocular Pose Estimation](https://mlsanity.com/paper/2207.07032/) (2022)
- [Bidirectional Semi-supervised Dual-branch CNN for Robust 3D Reconstruction of Stereo Endoscopic Images via Adaptive Cross and Parallel Supervisions](https://mlsanity.com/paper/2210.08291/) (2022)
- [Detecting the Sensing Area of A Laparoscopic Probe in Minimally Invasive Cancer Surgery](https://mlsanity.com/paper/2307.03662/) (2023)
- [3D endoscopic depth estimation using 3D surface-aware constraints](https://mlsanity.com/paper/2203.02131/) (2022)
- [A comprehensive survey on recent deep learning-based methods applied to surgical data](https://mlsanity.com/paper/2209.01435/) (2022)
- [3D Densification for Multi-Map Monocular VSLAM in Endoscopy](https://mlsanity.com/paper/2503.14346/) (2025)
- [Dense 3D Reconstruction Through Lidar: A Comparative Study on Ex-vivo Porcine Tissue](https://mlsanity.com/paper/2401.10709/) (2024)
- [3D Guidewire Shape Reconstruction from Monoplane Fluoroscopic Images](https://mlsanity.com/paper/2311.11209/) (2023)
- [High-fidelity Endoscopic Image Synthesis by Utilizing Depth-guided Neural Surfaces](https://mlsanity.com/paper/2404.13437/) (2024)
- [Semi-Supervised Adversarial Discriminative Domain Adaptation](https://mlsanity.com/paper/2109.13016/) (2021)
- [A Compacted Structure for Cross-domain learning on Monocular Depth and Flow Estimation](https://mlsanity.com/paper/2208.11993/) (2022)
- [Nested ResNet: A Vision-Based Method for Detecting the Sensing Area of a Drop-in Gamma Probe](https://mlsanity.com/paper/2410.23154/) (2024)

## Abstract (arXiv)

Dense depth estimation and 3D reconstruction of a surgical scene are crucial steps in computer assisted surgery. Recent work has shown that depth estimation from a stereo images pair could be solved with convolutional neural networks. However, most recent depth estimation models were trained on datasets with per-pixel ground truth. Such data is especially rare for laparoscopic imaging, making it hard to apply supervised depth estimation to real surgical applications. To overcome this limitation, we propose SADepth, a new self-supervised depth estimation method based on Generative Adversarial Networks. It consists of an encoder-decoder generator and a discriminator to incorporate geometry constraints during training. Multi-scale outputs from the generator help to solve the local minima caused by the photometric reprojection loss, while the adversarial learning improves the framework generation quality. Extensive experiments on two public datasets show that SADepth outperforms recent state-of-the-art unsupervised methods by a large margin, and reduces the gap between supervised and unsupervised depth estimation in laparoscopic images.