# Differentiable Rendering-based Pose Estimation for Surgical Robotic Instruments

arXiv: https://arxiv.org/abs/2503.05953 · HTML: https://mlsanity.com/paper/2503.05953/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Mar 2025.

## Built on by (most cited first, 7 of 7)

- [Efficient Surgical Robotic Instrument Pose Reconstruction in Real World Conditions Using Unified Feature Detection](https://mlsanity.com/paper/2510.03532/) (2025)
- [ToolTipNet: A Segmentation-Driven Deep Learning Baseline for Surgical Instrument Tip Detection](https://mlsanity.com/paper/2504.09700/) (2025)
- [SurgCalib: Gaussian Splatting-Based Hand-Eye Calibration for Robot-Assisted Minimally Invasive Surgery](https://mlsanity.com/paper/2603.08983/) (2026)
- [Real-time Rendering-based Surgical Instrument Tracking via Evolutionary Optimization](https://mlsanity.com/paper/2603.11404/) (2026)
- [MarUco: A Markerless 6D Pose Estimation Framework for Closed-Loop Control of Surgical Continuum Manipulators](https://mlsanity.com/paper/2602.16365/) (2026)
- [SurfSurg6D: Geometry Consistent Dense Correspondence for Textureless Surgical Instrument Pose Estimation](https://mlsanity.com/paper/2605.25598/) (2026)
- [Robust Surgical Robotic Instrument Tracking via Sequential Multi-Cue Fusion and Sim-to-Real Self-Training](https://mlsanity.com/paper/2610.05491/) (2026)

## Abstract (arXiv)

Robot pose estimation is a challenging and crucial task for vision-based surgical robotic automation. Typical robotic calibration approaches, however, are not applicable to surgical robots, such as the da Vinci Research Kit (dVRK), due to joint angle measurement errors from cable-drives and the partially visible kinematic chain. Hence, previous works in surgical robotic automation used tracking algorithms to estimate the pose of the surgical tool in real-time and compensate for the joint angle errors. However, a big limitation of these previous tracking works is the initialization step which relied on only keypoints and SolvePnP. In this work, we fully explore the potential of geometric primitives beyond just keypoints with differentiable rendering, cylinders, and construct a versatile pose matching pipeline in a novel pose hypothesis space. We demonstrate the state-of-the-art performance of our single-shot calibration method with both calibration consistency and real surgical tasks. As a result, this marker-less calibration approach proves to be a robust and generalizable initialization step for surgical tool tracking.