# SurgCalib: Gaussian Splatting-Based Hand-Eye Calibration for Robot-Assisted Minimally Invasive Surgery

arXiv: https://arxiv.org/abs/2603.08983 · HTML: https://mlsanity.com/paper/2603.08983/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 39 references that are in the map.
- First posted on arXiv 9 Mar 2026.

## Builds on (most cited first, 21 of 21)

- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [SurgicalSAM: Efficient Class Promptable Surgical Instrument Segmentation](https://mlsanity.com/paper/2308.08746/) (2023)
- [Camera-to-Robot Pose Estimation from a Single Image](https://mlsanity.com/paper/1911.09231/) (2019)
- [Single-view robot pose and joint angle estimation via render & compare](https://mlsanity.com/paper/2104.09359/) (2021)
- [Markerless Camera-to-Robot Pose Estimation via Self-supervised Sim-to-Real Transfer](https://mlsanity.com/paper/2302.14332/) (2023)
- [Differentiable Robot Rendering](https://mlsanity.com/paper/2410.13851/) (2024)
- [SurgPose: a Dataset for Articulated Robotic Surgical Tool Pose Estimation and Tracking](https://mlsanity.com/paper/2502.11534/) (2025)
- [Self-Calibrating Gaussian Splatting for Large Field of View Reconstruction](https://mlsanity.com/paper/2502.09563/) (2025)
- [Neural Lens Modeling](https://mlsanity.com/paper/2304.04848/) (2023)
- [Augmenting Efficient Real-time Surgical Instrument Segmentation in Video with Point Tracking and Segment Anything](https://mlsanity.com/paper/2403.08003/) (2024)
- [Differentiable Rendering-based Pose Estimation for Surgical Robotic Instruments](https://mlsanity.com/paper/2503.05953/) (2025)
- [Caveats on the first-generation da Vinci Research Kit: latent technical constraints and essential calibrations](https://mlsanity.com/paper/2210.13598/) (2022)
- [Video-based Surgical Tool-tip and Keypoint Tracking using Multi-frame Context-driven Deep Learning Models](https://mlsanity.com/paper/2501.18361/) (2025)
- [Instrument-Splatting: Controllable Photorealistic Reconstruction of Surgical Instruments Using Gaussian Splatting](https://mlsanity.com/paper/2503.04082/) (2025)
- [Efficient Surgical Robotic Instrument Pose Reconstruction in Real World Conditions Using Unified Feature Detection](https://mlsanity.com/paper/2510.03532/) (2025)
- [ROBUST-MIPS: A Combined Skeletal Pose and Instance Segmentation Dataset for Laparoscopic Surgical Instruments](https://mlsanity.com/paper/2508.21096/) (2025)
- [On-the-fly hand-eye calibration for the da Vinci surgical robot](https://mlsanity.com/paper/2601.14871/) (2026)
- [Kalib: Easy Hand-Eye Calibration with Reference Point Tracking](https://mlsanity.com/paper/2408.10562/) (2024)
- [ToolTipNet: A Segmentation-Driven Deep Learning Baseline for Surgical Instrument Tip Detection](https://mlsanity.com/paper/2504.09700/) (2025)

## Built on by (most cited first, 1 of 1)

- [Robust Surgical Robotic Instrument Tracking via Sequential Multi-Cue Fusion and Sim-to-Real Self-Training](https://mlsanity.com/paper/2610.05491/) (2026)

## Abstract (arXiv)

We present a Gaussian Splatting-based framework for hand-eye calibration of the da Vinci surgical robot. In a vision-guided robotic system, accurate estimation of the rigid transformation between the robot base and the camera frame is essential for reliable closed-loop control. For cable-driven surgical robots, this task faces unique challenges. The encoders of surgical instruments often produce inaccurate proprioceptive measurements due to cable stretch and backlash. Conventional hand-eye calibration approaches typically rely on known fiducial patterns and solve the AX = XB formulation. While effective, introducing additional markers into the operating room (OR) environment can violate sterility protocols and disrupt surgical workflows. In this study, we propose SurgCalib, an automatic, markerless framework that has the potential to be used in the OR. SurgCalib first initializes the pose of the surgical instrument using raw kinematic measurements and subsequently refines this pose through a two-phase optimization procedure under the RCM constraint within a Gaussian Splatting-based differentiable rendering pipeline. We evaluate the proposed method on the public dVRK benchmark, SurgPose. The results demonstrate average 2D tool-tip reprojection errors of 12.24 px (2.06 mm) and 11.33 px (1.9 mm), and 3D tool-tip Euclidean distance errors of 5.98 mm and 4.75 mm, for the left and right instruments, respectively.