# Real-time Dense Reconstruction of Tissue Surface from Stereo Optical Video

arXiv: https://arxiv.org/abs/2007.12623 · HTML: https://mlsanity.com/paper/2007.12623/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jul 2020.

## Built on by (most cited first, 17 of 17)

- [Neural Rendering for Stereo 3D Reconstruction of Deformable Tissues in Robotic Surgery](https://mlsanity.com/paper/2206.15255/) (2022)
- [Tracking and Mapping in Medical Computer Vision: A Review](https://mlsanity.com/paper/2310.11475/) (2023)
- [EndoGS: Deformable Endoscopic Tissues Reconstruction with Gaussian Splatting](https://mlsanity.com/paper/2401.11535/) (2024)
- [EndoGaussian: Real-time Gaussian Splatting for Dynamic Endoscopic Scene Reconstruction](https://mlsanity.com/paper/2401.12561/) (2024)
- [Efficient Deformable Tissue Reconstruction via Orthogonal Neural Plane](https://mlsanity.com/paper/2312.15253/) (2023)
- [EndoSparse: Real-Time Sparse View Synthesis of Endoscopic Scenes using Gaussian Splatting](https://mlsanity.com/paper/2407.01029/) (2024)
- [SurgicalGS: Dynamic 3D Gaussian Splatting for Accurate Robotic-Assisted Surgical Scene Reconstruction](https://mlsanity.com/paper/2410.09292/) (2024)
- [Stereo Dense Scene Reconstruction and Accurate Localization for Learning-Based Navigation of Laparoscope in Minimally Invasive Surgery](https://mlsanity.com/paper/2110.03912/) (2021)
- [Real-time Nonrigid Mosaicking of Laparoscopy Images](https://mlsanity.com/paper/2103.07414/) (2021)
- [Diff2DGS: Reliable Reconstruction of Occluded Surgical Scenes via 2D Gaussian Splatting](https://mlsanity.com/paper/2602.18314/) (2026)
- [Free-DyGS: Camera-Pose-Free Scene Reconstruction for Dynamic Surgical Videos with Gaussian Splatting](https://mlsanity.com/paper/2409.01003/) (2024)
- [A Disparity Refinement Framework for Learning-based Stereo Matching Methods in Cross-domain Setting for Laparoscopic Images](https://mlsanity.com/paper/2302.02294/) (2023)
- [MT3DNet: Multi-Task learning Network for 3D Surgical Scene Reconstruction](https://mlsanity.com/paper/2412.03928/) (2024)
- [ColorGS: High-fidelity Surgical Scene Reconstruction with Colored Gaussian Splatting](https://mlsanity.com/paper/2508.18696/) (2025)
- [SAGS: Self-Adaptive Alias-Free Gaussian Splatting for Dynamic Surgical Endoscopic Reconstruction](https://mlsanity.com/paper/2510.27318/) (2025)
- [NeVStereo: A NeRF-Driven NVS-Stereo Architecture for High-Fidelity 3D Tasks](https://mlsanity.com/paper/2602.05423/) (2026)
- [Monocular Endoscopic Tissue 3D Reconstruction with Multi-Level Geometry Regularization](https://mlsanity.com/paper/2602.20718/) (2026)

## Abstract (arXiv)

We propose an approach to reconstruct dense three-dimensional (3D) model of tissue surface from stereo optical videos in real-time, the basic idea of which is to first extract 3D information from video frames by using stereo matching, and then to mosaic the reconstructed 3D models. To handle the common low texture regions on tissue surfaces, we propose effective post-processing steps for the local stereo matching method to enlarge the radius of constraint, which include outliers removal, hole filling and smoothing. Since the tissue models obtained by stereo matching are limited to the field of view of the imaging modality, we propose a model mosaicking method by using a novel feature-based simultaneously localization and mapping (SLAM) method to align the models. Low texture regions and the varying illumination condition may lead to a large percentage of feature matching outliers. To solve this problem, we propose several algorithms to improve the robustness of SLAM, which mainly include (1) a histogram voting-based method to roughly select possible inliers from the feature matching results, (2) a novel 1-point RANSAC-based PnP algorithm called as DynamicR1PPnP to track the camera motion and (3) a GPU-based iterative closest points (ICP) and bundle adjustment (BA) method to refine the camera motion estimation results. Experimental results on ex- and in vivo data showed that the reconstructed 3D models have high resolution texture with an accuracy error of less than 2 mm. Most algorithms are highly parallelized for GPU computation, and the average runtime for processing one key frame is 76.3 ms on stereo images with 960x540 resolution.