mlsanityData to 14 Oct 2026

Self-Supervised Monocular Depth and Ego-Motion Estimation in Endoscopy: Appearance Flow to the Rescue

arXiv 2112.08122 ↗First posted 15 Dec 2021cs.CVcited by 72

Not on the map. Why?

Abstract (arXiv)

Recently, self-supervised learning technology has been applied to calculate depth and ego-motion from monocular videos, achieving remarkable performance in autonomous driving scenarios. One widely adopted assumption of depth and ego-motion self-supervised learning is that the image brightness remains constant within nearby frames. Unfortunately, the endoscopic scene does not meet this assumption because there are severe brightness fluctuations induced by illumination variations, non-Lambertian reflections and interreflections during data collection, and these brightness fluctuations inevitably deteriorate the depth and ego-motion estimation accuracy. In this work, we introduce a novel concept referred to as appearance flow to address the brightness inconsistency problem. The appearance flow takes into consideration any variations in the brightness pattern and enables us to develop a generalized dynamic image constraint. Furthermore, we build a unified self-supervised framework to estimate monocular depth and ego-motion simultaneously in endoscopic scenes, which comprises a structure module, a motion module, an appearance module and a correspondence module, to accurately reconstruct the appearance and calibrate the image brightness. Extensive experiments are conducted on the SCARED dataset and EndoSLAM dataset, and the proposed unified framework exceeds other self-supervised approaches by a large margin. To validate our framework's generalization ability on different patients and cameras, we train our model on SCARED but test it on the SERV-CT and Hamlyn datasets without any fine-tuning, and the superior results reveal its strong generalization ability. Code will be available at: urlhttps://github.com/ShuweiShao/AF-SfMLearner.

Key facts. Cited by 72 papers in the map (data through 14 Oct 2026). Builds on 45 of its 73 references that are in the map. First posted on arXiv 15 Dec 2021.

Builds on

45 of 73 references.

References · 45

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2016Pyramid Scene Parsing Networkcited by 2,479
  4. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
  5. 2015Spatial Transformer Networkscited by 1,429
  6. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,274
  7. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,059
  8. 2015ORB-SLAM: a Versatile and Accurate Monocular SLAM Systemcited by 988
  9. 2016Unsupervised Monocular Depth Estimation with Left-Right Consistencycited by 820
  10. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 782
  11. 2018Deep Ordinal Regression Network for Monocular Depth Estimationcited by 723
  12. 2018GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Posecited by 342
33 more
  1. 2019Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Videocited by 244
  2. 2018Competitive Collaboration: Joint Unsupervised Learning of Depth, Camera Motion, Optical Flow and Motion Segmentationcited by 222
  3. 2020D3VO: Deep Depth, Deep Pose and Deep Uncertainty for Monocular Visual Odometrycited by 206
  4. 2016Deep Feature Flow for Video Recognitioncited by 186
  5. 2016Single-Image Depth Perception in the Wildcited by 180
  6. 2018Unsupervised Learning of Depth and Ego-Motion from Monocular Video Using 3D Geometric Constraintscited by 179
  7. 2015Learning Depth from Single Monocular Images Using Deep Convolutional Neural Fieldscited by 169
  8. 2018Unsupervised Learning of Monocular Depth Estimation and Visual Odometry with Deep Feature Reconstructioncited by 159
  9. 2018Depth Prediction Without the Sensors: Leveraging Structure for Unsupervised Learning from Monocular Videoscited by 155
  10. 2020Feature-metric Loss for Self-supervised Learning of Depth and Egomotioncited by 140
  11. 2019Iterative Residual Refinement for Joint Optical Flow and Occlusion Estimationcited by 131
  12. 2019Self-supervised Learning with Geometric Constraints in Monocular Video: Connecting Flow, Depth, and Cameracited by 130
  13. 2020Self-supervised Monocular Trained Depth Estimation using Self-attention and Discrete Disparity Volumecited by 116
  14. 2021Stereo Correspondence and Reconstruction of Endoscopic Data Challengecited by 103
  15. 2017Occlusion Aware Unsupervised Learning of Optical Flowcited by 101
  16. 2018Every Pixel Counts ++: Joint Learning of Geometry and Motion with 3D Holistic Understandingcited by 101
  17. 2016Estimating Depth from Monocular Images as Classification Using Deep Fully Convolutional Residual Networkscited by 91
  18. 2021Endo-Depth-and-Motion: Reconstruction and Tracking in Endoscopic Videos using Depth Networks and Photometric Constraintscited by 81
  19. 2018Structured Attention Guided Convolutional Neural Fields for Monocular Depth Estimationcited by 78
  20. 2017Multi-Scale Continuous CRFs as Sequential Deep Networks for Monocular Depth Estimationcited by 70
  21. 2019Unsupervised 3D End-to-End Medical Image Registration with Volume Tweening Networkcited by 62
  22. 2020DeFeat-Net: General Monocular Depth via Simultaneous Unsupervised Representation Learningcited by 60
  23. 2017Unsupervised Reverse Domain Adaptation for Synthetic Medical Images via Adversarial Trainingcited by 57
  24. 2018LEGO: Learning Edge with Geometry all at Once by Watching Videoscited by 47
  25. 2019Dense Depth Estimation in Monocular Endoscopy with Self-supervised Learning Methodscited by 47
  26. 2017Deep Learning and Conditional Random Fields-based Depth Estimation and Topographical Reconstruction from Conventional Endoscopycited by 31
  27. 2019SLAM Endoscopy enhanced by adversarial depth predictioncited by 26
  28. 2019Unsupervised High-Resolution Depth Learning From Videos With Dual Networkscited by 26
  29. 2018Learning Depth from Single Images with Deep Neural Network Embedding Focal Lengthcited by 25
  30. 2019Deeply-supervised Knowledge Synergycited by 18
  31. 2021Test-Time Training for Deformable Multi-Scale Image Registrationcited by 17
  32. 2018Unsupervised Odometry and Depth Learning for Endoscopic Capsule Robotscited by 14
  33. 2018Dual CNN Models for Unsupervised Monocular Depth Estimationcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 4; 2023: 12; 2024: 23; 2025: 18; 2026: 15.

72 papers.

Sort:

By area:

  1. 2024Surgical-DINO: Adapter Learning of Foundation Models for Depth Estimation in Endoscopic Surgerycited by 52
  2. 2024Endo-4DGS: Endoscopic Monocular Scene Reconstruction with 4D Gaussian Splattingcited by 43
  3. 2024EndoDAC: Efficient Adapting Foundation Model for Self-Supervised Depth Estimation from Any Endoscopic Cameracited by 43
  4. 2024EndoGSLAM: Real-Time Dense Reconstruction and Tracking in Endoscopic Surgeries using Gaussian Splattingcited by 33
  5. 2025Scaling up self-supervised learning for improved surgical foundation modelscited by 25
  6. 2023URCDC-Depth: Uncertainty Rectified Cross-Distillation with CutFlip for Monocular Depth Estimationcited by 16
  7. 2025Endo3R: Unified Online Reconstruction from Dynamic Monocular Endoscopic Videocited by 16
  8. 2024Frontiers in Intelligent Colonoscopycited by 14
  9. 2024Depth Anything in Medical Images: A Comparative Studycited by 13
  10. 2024Transferring Relative Monocular Depth to Surgical Vision with Temporal Consistencycited by 13
  11. 2024EndoOmni: Zero-Shot Cross-Dataset Depth Estimation in Endoscopy by Robust Self-Learning from Noisy Labelscited by 12
  12. 2025Advancing Dense Endoscopic Reconstruction with Gaussian Splatting-driven Surface Normal-aware Tracking and Mappingcited by 11
38 more
  1. 2024MonoPCC: Photometric-invariant Cycle Constraint for Monocular Depth Estimation of Endoscopic Imagescited by 10
  2. 2023WS-SfMLearner: Self-supervised Monocular Depth and Ego-motion Estimation on Surgical Videos with Unknown Camera Parameterscited by 9
  3. 2023A geometry-aware deep network for depth estimation in monocular endoscopycited by 8
  4. 2024DARES: Depth Anything in Robotic Endoscopic Surgery with Self-supervised Vector-LoRA of the Foundation Modelcited by 8
  5. 2022C3Fusion: Consistent Contrastive Colon Fusion, Towards Deep SLAM in Colonoscopycited by 7
  6. 2022Multi-Scale Structural-aware Exposure Correction for Endoscopic Imagingcited by 6
  7. 2024DD-VNB: A Depth-based Dual-Loop Framework for Real-time Visually Navigated Bronchoscopycited by 6
  8. 2025Endo-FASt3r: Endoscopic Foundation model Adaptation for Structure from motioncited by 6
  9. 2025Mono3R: Exploiting Monocular Cues for Geometric 3D Reconstructioncited by 6
  10. 2023The Drunkard's Odometry: Estimating Camera Motion in Deforming Scenescited by 5
  11. 2023Task-guided Domain Gap Reduction for Monocular Depth Prediction in Endoscopycited by 5
  12. 2024Surgical Depth Anything: Depth Estimation for Surgical Scenes using Foundation Modelscited by 5
  13. 2024BodySLAM: A Generalized Monocular Visual SLAM Framework for Surgical Applicationscited by 4
  14. 2024Enhanced Scale-aware Depth Estimation for Monocular Endoscopic Scenes with Geometric Modelingcited by 4
  15. 2024Advancing Depth Anything Model for Unsupervised Monocular Depth Estimation in Endoscopycited by 4
  16. 2022A Novel Hybrid Endoscopic Dataset for Evaluating Machine Learning-based Photometric Image Enhancement Modelscited by 3
  17. 2025SHADeS: Self-supervised Monocular Depth Estimation Through Non-Lambertian Image Decompositioncited by 3
  18. 2023EndoDepthL: Lightweight Endoscopic Monocular Depth Estimation with CNN-Transformercited by 2
  19. 2024Neural Fields for 3D Tracking of Anatomy and Surgical Instruments in Monocular Laparoscopic Video Clipscited by 2
  20. 2024Depth-Aware Endoscopic Video Inpaintingcited by 2
  21. 2024EndoDepth: A Benchmark for Assessing Robustness in Endoscopic Depth Predictioncited by 2
  22. 2025ColonAdapter: Geometry Estimation Through Foundation Model Adaptation for Colonoscopycited by 2
  23. 2026BREATH-VL: Vision-Language-Guided 6-DoF Bronchoscopy Localization via Semantic-Geometric Fusioncited by 2
  24. 2026RealSynCol: a high-fidelity synthetic colon dataset for 3D reconstruction applicationscited by 2
  25. 2022Ego-motion Estimation Based on Fusion of Images and Eventscited by 1
  26. 2023HQDec: Self-Supervised Monocular Depth Estimation Based on a High-Quality Decodercited by 1
  27. 2023Unleashing the Power of Depth and Pose Estimation Neural Networks by Designing Compatible Endoscopic Imagescited by 1
  28. 2024Generalizing monocular colonoscopy image depth estimation by uncertainty-based global and local fusion networkcited by 1
  29. 2024Benchmarking Robustness of Endoscopic Depth Estimation with Synthetically Corrupted Datacited by 1
  30. 2024EndoPerfect: High-Accuracy Monocular Depth Estimation and 3D Reconstruction for Endoscopic Surgery via NeRF-Stereo Fusioncited by 1
  31. 2025EndoPBR: Material and Lighting Estimation for Photorealistic Surgical Simulations via Physically-based Renderingcited by 1
  32. 2025Occlusion-Aware Self-Supervised Monocular Depth Estimation for Weak-Texture Endoscopic Imagescited by 1
  33. 2025Endoscopic Depth Estimation Based on Deep Learning: A Surveycited by 1
  34. 2025EndoUFM: Utilizing Foundation Models for Monocular depth estimation of endoscopic imagescited by 1
  35. 2025More than Segmentation: Benchmarking SAM 3 for Segmentation, 3D Perception, and Reconstruction in Robotic Surgerycited by 1
  36. 20264D Monocular Surgical Reconstruction under Arbitrary Camera Motionscited by 1
  37. 2023SoftEnNet: Symbiotic Monocular Depth Estimation and Lumen Segmentation for Colonoscopy Endorobotscited by 0
  38. 2023Deep learning-based image exposure enhancement as a pre-processing for an accurate 3D colon surface reconstructioncited by 0

50 most cited of 72 All 72 (JSON)