# SurgicalGS: Dynamic 3D Gaussian Splatting for Accurate Robotic-Assisted Surgical Scene Reconstruction

arXiv: https://arxiv.org/abs/2410.09292 · HTML: https://mlsanity.com/paper/2410.09292/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 45 references that are in the map.
- First posted on arXiv 11 Oct 2024.

## Builds on (most cited first, 28 of 28)

- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data](https://mlsanity.com/paper/2401.10891/) (2024)
- [K-Planes: Explicit Radiance Fields in Space, Time, and Appearance](https://mlsanity.com/paper/2301.10241/) (2023)
- [HexPlane: A Fast Representation for Dynamic Scenes](https://mlsanity.com/paper/2301.09632/) (2023)
- [Fast Dynamic Radiance Fields with Time-Aware Neural Voxels](https://mlsanity.com/paper/2205.15285/) (2022)
- [Differentiable Surface Splatting for Point-based Geometry Processing](https://mlsanity.com/paper/1906.04173/) (2019)
- [A Survey on 3D Gaussian Splatting](https://mlsanity.com/paper/2401.03890/) (2024)
- [Consistent Video Depth Estimation](https://mlsanity.com/paper/2004.15021/) (2020)
- [Tensor4D : Efficient Neural 4D Decomposition for High-fidelity Dynamic Reconstruction and Rendering](https://mlsanity.com/paper/2211.11610/) (2022)
- [Atlas: End-to-End 3D Scene Reconstruction from Posed Images](https://mlsanity.com/paper/2003.10432/) (2020)
- [Revisiting Stereo Depth Estimation From a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2011.02910/) (2020)
- [Neural Rendering for Stereo 3D Reconstruction of Deformable Tissues in Robotic Surgery](https://mlsanity.com/paper/2206.15255/) (2022)
- [EndoSurf: Neural Surface Reconstruction of Deformable Tissues with Stereo Endoscope Videos](https://mlsanity.com/paper/2307.11307/) (2023)
- [Learning How To Robustly Estimate Camera Pose in Endoscopic Videos](https://mlsanity.com/paper/2304.08023/) (2023)
- [Normal Assisted Stereo Depth Estimation](https://mlsanity.com/paper/1911.10444/) (2019)
- [SuPer: A Surgical Perception Framework for Endoscopic Tissue Manipulation with Surgical Robotics](https://mlsanity.com/paper/1909.05405/) (2019)
- [E-DSSR: Efficient Dynamic Surgical Scene Reconstruction with Transformer-based Stereoscopic Depth Perception](https://mlsanity.com/paper/2107.00229/) (2021)
- [Neural LerPlane Representations for Fast 4D Reconstruction of Deformable Tissues](https://mlsanity.com/paper/2305.19906/) (2023)
- [Deform3DGS: Flexible Deformation for Fast Surgical Scene Reconstruction with Gaussian Splatting](https://mlsanity.com/paper/2405.17835/) (2024)
- [SurfelWarp: Efficient Non-Volumetric Single View Dynamic Reconstruction](https://mlsanity.com/paper/1904.13073/) (2019)
- [Dynamic Reconstruction of Deformable Soft-tissue with Stereo Scope in Minimal Invasive Surgery](https://mlsanity.com/paper/2003.10867/) (2020)
- [Self-Supervised Generative Adversarial Network for Depth Estimation in Laparoscopic Images](https://mlsanity.com/paper/2107.04644/) (2021)
- [Self-Supervised Depth Estimation in Laparoscopic Image using 3D Geometric Consistency](https://mlsanity.com/paper/2208.08407/) (2022)
- [Real-time Dense Reconstruction of Tissue Surface from Stereo Optical Video](https://mlsanity.com/paper/2007.12623/) (2020)
- [Deep Laparoscopic Stereo Matching with Transformers](https://mlsanity.com/paper/2207.12152/) (2022)
- [CathAction: A Benchmark for Endovascular Intervention Understanding](https://mlsanity.com/paper/2408.13126/) (2024)
- [Tracking Everything in Robotic-Assisted Surgery](https://mlsanity.com/paper/2409.19821/) (2024)

## Built on by (most cited first, 14 of 14)

- [EndoWave: 4D Gaussian Splatting with Rational Wavelet for Endoscopic Reconstruction](https://mlsanity.com/paper/2510.23087/) (2025)
- [Real-Time Spatio-Temporal Reconstruction of Dynamic Endoscopic Scenes with 4D Gaussian Splatting](https://mlsanity.com/paper/2411.01218/) (2024)
- [Diff2DGS: Reliable Reconstruction of Occluded Surgical Scenes via 2D Gaussian Splatting](https://mlsanity.com/paper/2602.18314/) (2026)
- [ColonSplat: Reconstruction of Peristaltic Motion in Colonoscopy with Dynamic Gaussian Splatting](https://mlsanity.com/paper/2603.06860/) (2026)
- [Towards Next-Generation Healthcare: A Survey of Medical Embodied AI for Perception, Decision-Making, and Action](https://mlsanity.com/paper/2606.15647/) (2026)
- [Gaussian Splatting-based Low-Rank Tensor Representation for Multi-Dimensional Image Recovery](https://mlsanity.com/paper/2511.14270/) (2025)
- [Nested ResNet: A Vision-Based Method for Detecting the Sensing Area of a Drop-in Gamma Probe](https://mlsanity.com/paper/2410.23154/) (2024)
- [Feature-EndoGaussian: Feature Distilled Gaussian Splatting in Surgical Deformable Scene Reconstruction](https://mlsanity.com/paper/2503.06161/) (2025)
- [The Dresden Dataset for 4D Reconstruction of Non-Rigid Abdominal Surgical Scenes](https://mlsanity.com/paper/2603.02985/) (2026)
- [SurgCUT3R: Surgical Scene-Aware Continuous Understanding of Temporal 3D Representation](https://mlsanity.com/paper/2603.06971/) (2026)
- [3D Consistency Optimization for Self-Supervised Monocular Video Depth Estimation](https://mlsanity.com/paper/2606.15681/) (2026)
- [DeGenseGS: Geometrically and Semantically Decoupled Surgical Scene Understanding in 4D Gaussian Splatting](https://mlsanity.com/paper/2607.04761/) (2026)
- [Endo-TSR: Temporal Spectral Modeling of Appearance and Motion for Endoscopic Reconstruction](https://mlsanity.com/paper/2609.32399/) (2026)
- [SurgGMF: Fully Causal Gaussian Motion Forecasting for Anticipatory Surgical Scene Rendering](https://mlsanity.com/paper/2609.34733/) (2026)

## Abstract (arXiv)

Accurate 3D reconstruction of dynamic surgical scenes from endoscopic video is essential for robotic-assisted surgery. While recent 3D Gaussian Splatting methods have shown promise in achieving high-quality reconstructions with fast rendering speeds, their use of inverse depth loss functions compresses depth variations. This can lead to a loss of fine geometric details, limiting their ability to capture precise 3D geometry and effectiveness in intraoperative application. To address these challenges, we present SurgicalGS, a dynamic 3D Gaussian Splatting framework specifically designed for surgical scene reconstruction with improved geometric accuracy. Our approach first initialises a Gaussian point cloud using depth priors, employing binary motion masks to identify pixels with significant depth variations and fusing point clouds from depth maps across frames for initialisation. We use the Flexible Deformation Model to represent dynamic scene and introduce a normalised depth regularisation loss along with an unsupervised depth smoothness constraint to ensure more accurate geometric reconstruction. Extensive experiments on two real surgical datasets demonstrate that SurgicalGS achieves state-of-the-art reconstruction quality, especially in terms of accurate geometry, advancing the usability of 3D Gaussian Splatting in robotic-assisted surgery.