# WS-SfMLearner: Self-supervised Monocular Depth and Ego-motion Estimation on Surgical Videos with Unknown Camera Parameters

arXiv: https://arxiv.org/abs/2308.11776 · HTML: https://mlsanity.com/paper/2308.11776/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 40 references that are in the map.
- First posted on arXiv 22 Aug 2023.

## Builds on (most cited first, 30 of 31)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Unsupervised Learning of Depth and Ego-Motion from Video](https://mlsanity.com/paper/1704.07813/) (2017)
- [Deep Ordinal Regression Network for Monocular Depth Estimation](https://mlsanity.com/paper/1806.02446/) (2018)
- [Semantic Scene Completion from a Single Depth Image](https://mlsanity.com/paper/1611.08974/) (2016)
- [Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Video](https://mlsanity.com/paper/1908.10553/) (2019)
- [The Temporal Opportunist: Self-Supervised Multi-Frame Monocular Depth](https://mlsanity.com/paper/2104.14540/) (2021)
- [Depth from Videos in the Wild: Unsupervised Monocular Depth Learning from Unknown Cameras](https://mlsanity.com/paper/1904.04998/) (2019)
- [2017 Robotic Instrument Segmentation Challenge](https://mlsanity.com/paper/1902.06426/) (2019)
- [Feature-metric Loss for Self-supervised Learning of Depth and Egomotion](https://mlsanity.com/paper/2007.10603/) (2020)
- [Self-supervised Monocular Trained Depth Estimation using Self-attention and Discrete Disparity Volume](https://mlsanity.com/paper/2003.13951/) (2020)
- [HR-Depth: High Resolution Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/2012.07356/) (2020)
- [Stereo Correspondence and Reconstruction of Endoscopic Data Challenge](https://mlsanity.com/paper/2101.01133/) (2021)
- [Estimating Depth from Monocular Images as Classification Using Deep Fully Convolutional Residual Networks](https://mlsanity.com/paper/1605.02305/) (2016)
- [Self-Supervised Monocular Depth and Ego-Motion Estimation in Endoscopy: Appearance Flow to the Rescue](https://mlsanity.com/paper/2112.08122/) (2021)
- [Multi-Scale Continuous CRFs as Sequential Deep Networks for Monocular Depth Estimation](https://mlsanity.com/paper/1704.02157/) (2017)
- [Unsupervised Scale-consistent Depth Learning from Video](https://mlsanity.com/paper/2105.11610/) (2021)
- [DeFeat-Net: General Monocular Depth via Simultaneous Unsupervised Representation Learning](https://mlsanity.com/paper/2003.13446/) (2020)
- [LEGO: Learning Edge with Geometry all at Once by Watching Videos](https://mlsanity.com/paper/1803.05648/) (2018)
- [CaraNet: Context Axial Reverse Attention Network for Segmentation of Small Medical Objects](https://mlsanity.com/paper/2301.13366/) (2023)
- [DC-UNet: Rethinking the U-Net Architecture with Dual Channel Efficient CNN for Medical Images Segmentation](https://mlsanity.com/paper/2006.00414/) (2020)
- [Min-Max Similarity: A Contrastive Semi-Supervised Deep Learning Network for Surgical Tools Segmentation](https://mlsanity.com/paper/2203.15177/) (2022)
- [Learning from a tiny dataset of manual annotations: a teacher/student approach for surgical phase recognition](https://mlsanity.com/paper/1812.00033/) (2018)
- [Self-Supervised Generative Adversarial Network for Depth Estimation in Laparoscopic Images](https://mlsanity.com/paper/2107.04644/) (2021)
- [Transformers in Self-Supervised Monocular Depth Estimation with Unknown Camera Intrinsics](https://mlsanity.com/paper/2202.03131/) (2022)
- [Unsupervised Odometry and Depth Learning for Endoscopic Capsule Robots](https://mlsanity.com/paper/1803.01047/) (2018)
- [CFPNet: Channel-wise Feature Pyramid for Real-Time Semantic Segmentation](https://mlsanity.com/paper/2103.12212/) (2021)
- [Intuitive Surgical SurgToolLoc and SurgVU Challenges Results: 2022-2025](https://mlsanity.com/paper/2305.07152/) (2023)
- [False Negative/Positive Control for SAM on Noisy Medical Images](https://mlsanity.com/paper/2308.10382/) (2023)
- [CamLessMonoDepth: Monocular Depth Estimation with Unknown Camera Parameters](https://mlsanity.com/paper/2110.14347/) (2021)

## Built on by (most cited first, 9 of 9)

- [Transferring Relative Monocular Depth to Surgical Vision with Temporal Consistency](https://mlsanity.com/paper/2403.06683/) (2024)
- [NeuroBOLT: Resting-state EEG-to-fMRI Synthesis with Multi-dimensional Feature Mapping](https://mlsanity.com/paper/2410.05341/) (2024)
- [Surgical Scene Understanding in the Era of Foundation AI Models: A Comprehensive Review](https://mlsanity.com/paper/2502.14886/) (2025)
- [Surgical Depth Anything: Depth Estimation for Surgical Scenes using Foundation Models](https://mlsanity.com/paper/2410.07434/) (2024)
- [SHADeS: Self-supervised Monocular Depth Estimation Through Non-Lambertian Image Decomposition](https://mlsanity.com/paper/2502.12994/) (2025)
- [ColonCrafter: A Depth Estimation Model for Colonoscopy Videos Using Diffusion Priors](https://mlsanity.com/paper/2509.13525/) (2025)
- [Learning to Efficiently Adapt Foundation Models for Self-Supervised Endoscopic 3D Scene Reconstruction from Any Cameras](https://mlsanity.com/paper/2503.15917/) (2025)
- [DaRePlane: Direction-aware Representations for Dynamic Scene Reconstruction](https://mlsanity.com/paper/2410.14169/) (2024)
- [Confidence-aware Monocular Depth Estimation for Minimally Invasive Surgery](https://mlsanity.com/paper/2603.03571/) (2026)

## Abstract (arXiv)

Depth estimation in surgical video plays a crucial role in many image-guided surgery procedures. However, it is difficult and time consuming to create depth map ground truth datasets in surgical videos due in part to inconsistent brightness and noise in the surgical scene. Therefore, building an accurate and robust self-supervised depth and camera ego-motion estimation system is gaining more attention from the computer vision community. Although several self-supervision methods alleviate the need for ground truth depth maps and poses, they still need known camera intrinsic parameters, which are often missing or not recorded. Moreover, the camera intrinsic prediction methods in existing works depend heavily on the quality of datasets. In this work, we aimed to build a self-supervised depth and ego-motion estimation system which can predict not only accurate depth maps and camera pose, but also camera intrinsic parameters. We proposed a cost-volume-based supervision manner to give the system auxiliary supervision for camera parameters prediction. The experimental results showed that the proposed method improved the accuracy of estimated camera parameters, ego-motion, and depth estimation.