# A Quantitative Evaluation of Dense 3D Reconstruction of Sinus Anatomy from Monocular Endoscopic Video

arXiv: https://arxiv.org/abs/2310.14364 · HTML: https://mlsanity.com/paper/2310.14364/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 35 references that are in the map.
- First posted on arXiv 22 Oct 2023.

## Builds on (most cited first, 9 of 9)

- [Unsupervised Learning of Depth and Ego-Motion from Video](https://mlsanity.com/paper/1704.07813/) (2017)
- [Dense Depth Estimation in Monocular Endoscopy with Self-supervised Learning Methods](https://mlsanity.com/paper/1902.07766/) (2019)
- [DefSLAM: Tracking and Mapping of Deforming Scenes from Monocular Sequences](https://mlsanity.com/paper/1908.08918/) (2019)
- [MIS-SLAM: Real-time Large Scale Dense Deformable SLAM System in Minimal Invasive Surgery Based on Heterogeneous Computing](https://mlsanity.com/paper/1803.02009/) (2018)
- [SAGE: SLAM with Appearance and Geometry Prior for Endoscopy](https://mlsanity.com/paper/2202.09487/) (2022)
- [SLAM Endoscopy enhanced by adversarial depth prediction](https://mlsanity.com/paper/1907.00283/) (2019)
- [Reconstructing Sinus Anatomy from Endoscopic Video -- Towards a Radiation-free Approach for Quantitative Longitudinal Assessment](https://mlsanity.com/paper/2003.08502/) (2020)
- [Extremely Dense Point Correspondences using a Learned Feature Descriptor](https://mlsanity.com/paper/2003.00619/) (2020)
- [Sparse-then-Dense Alignment based 3D Map Reconstruction Method for Endoscopic Capsule Robots](https://mlsanity.com/paper/1708.09740/) (2017)

## Built on by (most cited first, 2 of 2)

- [An Endoscopic Chisel: Intraoperative Imaging Carves 3D Anatomical Models](https://mlsanity.com/paper/2402.11840/) (2024)
- [EndoPerfect: High-Accuracy Monocular Depth Estimation and 3D Reconstruction for Endoscopic Surgery via NeRF-Stereo Fusion](https://mlsanity.com/paper/2410.04041/) (2024)

## Abstract (arXiv)

Generating accurate 3D reconstructions from endoscopic video is a promising avenue for longitudinal radiation-free analysis of sinus anatomy and surgical outcomes. Several methods for monocular reconstruction have been proposed, yielding visually pleasant 3D anatomical structures by retrieving relative camera poses with structure-from-motion-type algorithms and fusion of monocular depth estimates. However, due to the complex properties of the underlying algorithms and endoscopic scenes, the reconstruction pipeline may perform poorly or fail unexpectedly. Further, acquiring medical data conveys additional challenges, presenting difficulties in quantitatively benchmarking these models, understanding failure cases, and identifying critical components that contribute to their precision. In this work, we perform a quantitative analysis of a self-supervised approach for sinus reconstruction using endoscopic sequences paired with optical tracking and high-resolution computed tomography acquired from nine ex-vivo specimens. Our results show that the generated reconstructions are in high agreement with the anatomy, yielding an average point-to-mesh error of 0.91 mm between reconstructions and CT segmentations. However, in a point-to-point matching scenario, relevant for endoscope tracking and navigation, we found average target registration errors of 6.58 mm. We identified that pose and depth estimation inaccuracies contribute equally to this error and that locally consistent sequences with shorter trajectories generate more accurate reconstructions. These results suggest that achieving global consistency between relative camera poses and estimated depths with the anatomy is essential. In doing so, we can ensure proper synergy between all components of the pipeline for improved reconstructions that will facilitate clinical application of this innovative technology.