# Self-Supervised Siamese Learning on Stereo Image Pairs for Depth Estimation in Robotic Surgery

arXiv: https://arxiv.org/abs/1705.08260 · HTML: https://mlsanity.com/paper/1705.08260/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 May 2017.

## Built on by (most cited first, 30 of 34)

- [Transformers in Medical Imaging: A Survey](https://mlsanity.com/paper/2201.09873/) (2022)
- [Stereo Correspondence and Reconstruction of Endoscopic Data Challenge](https://mlsanity.com/paper/2101.01133/) (2021)
- [Colonoscopy 3D Video Dataset with Paired Depth from 2D-3D Registration](https://mlsanity.com/paper/2206.08903/) (2022)
- [E-DSSR: Efficient Dynamic Surgical Scene Reconstruction with Transformer-based Stereoscopic Depth Perception](https://mlsanity.com/paper/2107.00229/) (2021)
- [ContIG: Self-supervised Multimodal Contrastive Learning for Medical Imaging with Genetics](https://mlsanity.com/paper/2111.13424/) (2021)
- [Tracking and Mapping in Medical Computer Vision: A Review](https://mlsanity.com/paper/2310.11475/) (2023)
- [Self-Supervised Generative Adversarial Network for Depth Estimation in Laparoscopic Images](https://mlsanity.com/paper/2107.04644/) (2021)
- [Semantic-SuPer: A Semantic-aware Surgical Perception Framework for Endoscopic Tissue Identification, Reconstruction, and Tracking](https://mlsanity.com/paper/2210.16674/) (2022)
- [SurgT challenge: Benchmark of Soft-Tissue Trackers for Robotic Surgery](https://mlsanity.com/paper/2302.03022/) (2023)
- [Transferring Relative Monocular Depth to Surgical Vision with Temporal Consistency](https://mlsanity.com/paper/2403.06683/) (2024)
- [EndoOmni: Zero-Shot Cross-Dataset Depth Estimation in Endoscopy by Robust Self-Learning from Noisy Labels](https://mlsanity.com/paper/2409.05442/) (2024)
- [A Mixed-Supervision Multilevel GAN Framework for Image Quality Enhancement](https://mlsanity.com/paper/2106.15575/) (2021)
- [A Review of Predictive and Contrastive Self-supervised Learning for Medical Images](https://mlsanity.com/paper/2302.05043/) (2023)
- [Deep Laparoscopic Stereo Matching with Transformers](https://mlsanity.com/paper/2207.12152/) (2022)
- [SAMSNeRF: Segment Anything Model (SAM) Guides Dynamic Surgical Scene Reconstruction by Neural Radiance Field (NeRF)](https://mlsanity.com/paper/2308.11774/) (2023)
- [Surgical Scene Understanding in the Era of Foundation AI Models: A Comprehensive Review](https://mlsanity.com/paper/2502.14886/) (2025)
- [Bidirectional Semi-supervised Dual-branch CNN for Robust 3D Reconstruction of Stereo Endoscopic Images via Adaptive Cross and Parallel Supervisions](https://mlsanity.com/paper/2210.08291/) (2022)
- [TiDy-PSFs: Computational Imaging with Time-Averaged Dynamic Point-Spread-Functions](https://mlsanity.com/paper/2303.17583/) (2023)
- [Detecting the Sensing Area of A Laparoscopic Probe in Minimally Invasive Cancer Surgery](https://mlsanity.com/paper/2307.03662/) (2023)
- [BASED: Bundle-Adjusting Surgical Endoscopic Dynamic Video Reconstruction using Neural Radiance Fields](https://mlsanity.com/paper/2309.15329/) (2023)
- [BDIS: Bayesian Dense Inverse Searching Method for Real-Time Stereo Surgical Image Matching](https://mlsanity.com/paper/2205.03133/) (2022)
- [LSD3K: A Benchmark for Smoke Removal from Laparoscopic Surgery Images](https://mlsanity.com/paper/2407.13132/) (2024)
- [Bayesian dense inverse searching algorithm for real-time stereo matching in minimally invasive surgery](https://mlsanity.com/paper/2106.07136/) (2021)
- [A comprehensive survey on recent deep learning-based methods applied to surgical data](https://mlsanity.com/paper/2209.01435/) (2022)
- [SimEndoGS: Efficient Data-driven Scene Simulation using Robotic Surgery Videos via Physics-embedded 3D Gaussians](https://mlsanity.com/paper/2405.00956/) (2024)
- [Dilated Fully Convolutional Neural Network for Depth Estimation from a Single Image](https://mlsanity.com/paper/2103.07570/) (2021)
- [Zero-shot Monocular Metric Depth for Endoscopic Images](https://mlsanity.com/paper/2509.18642/) (2025)
- [Redefining the Laparoscopic Spatial Sense: AI-based Intra- and Postoperative Measurement from Stereoimages](https://mlsanity.com/paper/2311.09744/) (2023)
- [Endo-TTAP: Robust Endoscopic Tissue Tracking via Multi-Facet Guided Attention and Hybrid Flow-point Supervision](https://mlsanity.com/paper/2503.22394/) (2025)
- [Endoscopic Depth Estimation Based on Deep Learning: A Survey](https://mlsanity.com/paper/2507.20881/) (2025)

## Abstract (arXiv)

Robotic surgery has become a powerful tool for performing minimally invasive procedures, providing advantages in dexterity, precision, and 3D vision, over traditional surgery. One popular robotic system is the da Vinci surgical platform, which allows preoperative information to be incorporated into live procedures using Augmented Reality (AR). Scene depth estimation is a prerequisite for AR, as accurate registration requires 3D correspondences between preoperative and intraoperative organ models. In the past decade, there has been much progress on depth estimation for surgical scenes, such as using monocular or binocular laparoscopes [1,2]. More recently, advances in deep learning have enabled depth estimation via Convolutional Neural Networks (CNNs) [3], but training requires a large image dataset with ground truth depths. Inspired by [4], we propose a deep learning framework for surgical scene depth estimation using self-supervision for scalable data acquisition. Our framework consists of an autoencoder for depth prediction, and a differentiable spatial transformer for training the autoencoder on stereo image pairs without ground truth depths. Validation was conducted on stereo videos collected in robotic partial nephrectomy.