# PR-ENDO: Physically Based Relightable Gaussian Splatting for Endoscopy

arXiv: https://arxiv.org/abs/2411.12510 · HTML: https://mlsanity.com/paper/2411.12510/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 55 references that are in the map.
- First posted on arXiv 19 Nov 2024.

## Builds on (most cited first, 30 of 30)

- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021)
- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [NeRF++: Analyzing and Improving Neural Radiance Fields](https://mlsanity.com/paper/2010.07492/) (2020)
- [Direct Sparse Odometry](https://mlsanity.com/paper/1607.02565/) (2016)
- [GaussianShader: 3D Gaussian Splatting with Shading Functions for Reflective Surfaces](https://mlsanity.com/paper/2311.17977/) (2023)
- [GS-IR: 3D Gaussian Splatting for Inverse Rendering](https://mlsanity.com/paper/2311.16473/) (2023)
- [Relightable 3D Gaussians: Realistic Point Cloud Relighting with BRDF Decomposition and Ray Tracing](https://mlsanity.com/paper/2311.16043/) (2023)
- [Neural Rendering for Stereo 3D Reconstruction of Deformable Tissues in Robotic Surgery](https://mlsanity.com/paper/2206.15255/) (2022)
- [GIR: 3D Gaussian Inverse Rendering for Relightable Scene Factorization](https://mlsanity.com/paper/2312.05133/) (2023)
- [Colonoscopy 3D Video Dataset with Paired Depth from 2D-3D Registration](https://mlsanity.com/paper/2206.08903/) (2022)
- [GaMeS: Mesh-Based Adapting and Modification of Gaussian Splatting](https://mlsanity.com/paper/2402.01459/) (2024)
- [EndoSurf: Neural Surface Reconstruction of Deformable Tissues with Stereo Endoscope Videos](https://mlsanity.com/paper/2307.11307/) (2023)
- [3D Gaussian Splatting with Deferred Reflection](https://mlsanity.com/paper/2404.18454/) (2024)
- [GS^3: Efficient Relighting with Triple Gaussian Splatting](https://mlsanity.com/paper/2410.11419/) (2024)
- [Endo-4DGS: Endoscopic Monocular Scene Reconstruction with 4D Gaussian Splatting](https://mlsanity.com/paper/2401.16416/) (2024)
- [GI-GS: Global Illumination Decomposition on Gaussian Splatting for Inverse Rendering](https://mlsanity.com/paper/2410.02619/) (2024)
- [DeferredGS: Decoupled and Editable Gaussian Splatting with Deferred Shading](https://mlsanity.com/paper/2404.09412/) (2024)
- [EndoGSLAM: Real-Time Dense Reconstruction and Tracking in Endoscopic Surgeries using Gaussian Splatting](https://mlsanity.com/paper/2403.15124/) (2024)
- [HVTR: Hybrid Volumetric-Textural Rendering for Human Avatars](https://mlsanity.com/paper/2112.10203/) (2021)
- [Gaussian Splatting with NeRF-based Color and Opacity](https://mlsanity.com/paper/2312.13729/) (2023)
- [SAGE: SLAM with Appearance and Geometry Prior for Endoscopy](https://mlsanity.com/paper/2202.09487/) (2022)
- [VMesh: Hybrid Volume-Mesh Representation for Efficient View Synthesis](https://mlsanity.com/paper/2303.16184/) (2023)
- [LightNeuS: Neural Surface Reconstruction in Endoscopy using Illumination Decline](https://mlsanity.com/paper/2309.02777/) (2023)
- [LumiGauss: Relightable Gaussian Splatting in the Wild](https://mlsanity.com/paper/2408.04474/) (2024)
- [Gaussian Pancakes: Geometrically-Regularized 3D Gaussian Splatting for Realistic Endoscopic Reconstruction](https://mlsanity.com/paper/2404.06128/) (2024)
- [EndoSparse: Real-Time Sparse View Synthesis of Endoscopic Scenes using Gaussian Splatting](https://mlsanity.com/paper/2407.01029/) (2024)
- [SLAM based Quasi Dense Reconstruction For Minimally Invasive Surgery Scenes](https://mlsanity.com/paper/1705.09107/) (2017)
- [Colonoscopy Navigation using End-to-End Deep Visuomotor Control: A User Study](https://mlsanity.com/paper/2206.15086/) (2022)

## Built on by (most cited first, 5 of 5)

- [ColonSplat: Reconstruction of Peristaltic Motion in Colonoscopy with Dynamic Gaussian Splatting](https://mlsanity.com/paper/2603.06860/) (2026)
- [Moving Light Adaptive Colonoscopy Reconstruction via Illumination-Attenuation-Aware 3D Gaussian Splatting](https://mlsanity.com/paper/2510.18739/) (2025)
- [Instrument-Splatting++: Towards Controllable Surgical Instrument Digital Twin Using Gaussian Splatting](https://mlsanity.com/paper/2603.22792/) (2026)
- [Gastroendoscopy View Synthesis: A New Real Dataset and Evaluation](https://mlsanity.com/paper/2606.25427/) (2026)
- [EndoMD-SLAM: Endoscopic Gaussian Splatting SLAM under Optical Degradation with Memory and Static-Transient Decomposition](https://mlsanity.com/paper/2608.08949/) (2026)

## Abstract (arXiv)

Endoluminal endoscopic procedures are essential for diagnosing colorectal cancer and other severe conditions in the digestive tract, urogenital system, and airways. 3D reconstruction and novel-view synthesis from endoscopic images are promising tools for enhancing diagnosis. Moreover, integrating physiological deformations and interaction with the endoscope enables the development of simulation tools from real video data. However, constrained camera trajectories and view-dependent lighting create artifacts, leading to inaccurate or overfitted reconstructions. We present PR-ENDO, a novel 3D reconstruction framework leveraging the unique property of endoscopic imaging, where a single light source is closely aligned with the camera. Our method separates light effects from tissue properties. PR-ENDO enhances 3D Gaussian Splatting with a physically based relightable model. We boost the traditional light transport formulation with a specialized MLP capturing complex light-related effects while ensuring reduced artifacts and better generalization across novel views. PR-ENDO achieves superior reconstruction quality compared to baseline methods on both public and in-house datasets. Unlike existing approaches, PR-ENDO enables tissue modifications while preserving a physically accurate response to light, making it closer to real-world clinical use.