# MedGS: Gaussian Splatting for Multi-Modal 3D Medical Imaging

arXiv: https://arxiv.org/abs/2509.16806 · HTML: https://mlsanity.com/paper/2509.16806/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 33 references that are in the map.
- First posted on arXiv 20 Sep 2025.

## Builds on (most cited first, 16 of 16)

- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Structure-Aware Sparse-View X-ray 3D Reconstruction](https://mlsanity.com/paper/2311.10959/) (2023)
- [R2-Gaussian: Rectifying Radiative Gaussian Splatting for Tomographic Reconstruction](https://mlsanity.com/paper/2405.20693/) (2024)
- [TotalSegmentator MRI: Robust Sequence-independent Segmentation of Multiple Anatomic Structures in MRI](https://mlsanity.com/paper/2405.19492/) (2024)
- [Radiative Gaussian Splatting for Efficient X-ray Novel View Synthesis](https://mlsanity.com/paper/2403.04116/) (2024)
- [Ultra-NeRF: Neural Radiance Fields for Ultrasound Imaging](https://mlsanity.com/paper/2301.10520/) (2023)
- [CortexODE: Learning Cortical Surface Reconstruction by Neural ODEs](https://mlsanity.com/paper/2202.08329/) (2022)
- [Neural Implicit Surface Reconstruction of Freehand 3D Ultrasound Volume with Geometric Constraints](https://mlsanity.com/paper/2401.05915/) (2024)
- [Neural deformation fields for template-based reconstruction of cortical surfaces from MRI](https://mlsanity.com/paper/2401.12938/) (2024)
- [Image-to-Graph Convolutional Network for Deformable Shape Reconstruction from a Single Projection Image](https://mlsanity.com/paper/2108.12533/) (2021)
- [VeGaS: Video Gaussian Splatting](https://mlsanity.com/paper/2411.11024/) (2024)
- [DefCor-Net: Physics-Aware Ultrasound Deformation Correction](https://mlsanity.com/paper/2308.03865/) (2023)
- [Are Pixel-Wise Metrics Reliable for Sparse-View Computed Tomography Reconstruction?](https://mlsanity.com/paper/2506.02093/) (2025)
- [A Narrative Review of Image Processing Techniques Related to Prostate Ultrasound](https://mlsanity.com/paper/2407.00678/) (2024)

## Built on by (most cited first, 3 of 3)

- [Medical Scene Reconstruction and Segmentation based on 3D Gaussian Representation](https://mlsanity.com/paper/2512.22800/) (2025)
- [Rendering Novel Views of MRI Using 3D Gaussian Splatting](https://mlsanity.com/paper/2606.26236/) (2026)
- [COGENT: Counterfactual Gaussian Explanations for Volumetric Medical Images](https://mlsanity.com/paper/2608.11422/) (2026)

## Abstract (arXiv)

Endoluminal endoscopic procedures are essential for diagnosing colorectal cancer and other severe conditions in the digestive tract, urogenital system, and airways. 3D reconstruction and novel-view synthesis from endoscopic images are promising tools for enhancing diagnosis. Moreover, integrating physiological deformations and interaction with the endoscope enables the development of simulation tools from real video data. However, constrained camera trajectories and view-dependent lighting create artifacts, leading to inaccurate or overfitted reconstructions. We present MedGS, a novel 3D reconstruction framework leveraging the unique property of endoscopic imaging, where a single light source is closely aligned with the camera. Our method separates light effects from tissue properties. MedGS enhances 3D Gaussian Splatting with a physically based relightable model. We boost the traditional light transport formulation with a specialized MLP capturing complex light-related effects while ensuring reduced artifacts and better generalization across novel views. MedGS achieves superior reconstruction quality compared to baseline methods on both public and in-house datasets. Unlike existing approaches, MedGS enables tissue modifications while preserving a physically accurate response to light, making it closer to real-world clinical use. Repository: https://github.com/gmum/MedGS