# EndoPerfect: High-Accuracy Monocular Depth Estimation and 3D Reconstruction for Endoscopic Surgery via NeRF-Stereo Fusion

arXiv: https://arxiv.org/abs/2410.04041 · HTML: https://mlsanity.com/paper/2410.04041/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 30 references that are in the map.
- First posted on arXiv 5 Oct 2024.

## Builds on (most cited first, 16 of 16)

- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data](https://mlsanity.com/paper/2401.10891/) (2024)
- [Depth-supervised NeRF: Fewer Views and Faster Training for Free](https://mlsanity.com/paper/2107.02791/) (2021)
- [Nerfstudio: A Modular Framework for Neural Radiance Field Development](https://mlsanity.com/paper/2302.04264/) (2023)
- [Depth Pro: Sharp Monocular Metric Depth in Less Than a Second](https://mlsanity.com/paper/2410.02073/) (2024)
- [Selective-Stereo: Adaptive Frequency Information Selection for Stereo Matching](https://mlsanity.com/paper/2403.00486/) (2024)
- [Self-Supervised Monocular Depth and Ego-Motion Estimation in Endoscopy: Appearance Flow to the Rescue](https://mlsanity.com/paper/2112.08122/) (2021)
- [Surgical-DINO: Adapter Learning of Foundation Models for Depth Estimation in Endoscopic Surgery](https://mlsanity.com/paper/2401.06013/) (2024)
- [MVSFormer++: Revealing the Devil in Transformer's Details for Multi-View Stereo](https://mlsanity.com/paper/2401.11673/) (2024)
- [Dense Depth Estimation in Monocular Endoscopy with Self-supervised Learning Methods](https://mlsanity.com/paper/1902.07766/) (2019)
- [EndoDAC: Efficient Adapting Foundation Model for Self-Supervised Depth Estimation from Any Endoscopic Camera](https://mlsanity.com/paper/2405.08672/) (2024)
- [Reconstructing Sinus Anatomy from Endoscopic Video -- Towards a Radiation-free Approach for Quantitative Longitudinal Assessment](https://mlsanity.com/paper/2003.08502/) (2020)
- [A Quantitative Evaluation of Dense 3D Reconstruction of Sinus Anatomy from Monocular Endoscopic Video](https://mlsanity.com/paper/2310.14364/) (2023)

## Built on by (most cited first, 1 of 1)

- [NeVStereo: A NeRF-Driven NVS-Stereo Architecture for High-Fidelity 3D Tasks](https://mlsanity.com/paper/2602.05423/) (2026)

## Abstract (arXiv)

In endoscopic sinus surgery (ESS), intraoperative CT (iCT) offers valuable intraoperative assessment but is constrained by slow deployment and radiation exposure, limiting its clinical utility. Endoscope-based monocular 3D reconstruction is a promising alternative; however, existing techniques often struggle to achieve the submillimeter precision required for dense reconstruction. In this work, we propose an iterative online learning approach that leverages Neural Radiance Fields (NeRF) as an intermediate representation, enabling monocular depth estimation and 3D reconstruction without relying on prior medical data. Our method attains a point-to-point accuracy below 0.5 mm, with a demonstrated theoretical depth accuracy of 0.125 ± 0.443 mm. We validate our approach across synthetic, phantom, and real endoscopic scenarios, confirming its accuracy and reliability. These results underscore the potential of our pipeline as an iCT alternative, meeting the demanding submillimeter accuracy standards required in ESS.