# SurgiPose: Estimating Surgical Tool Kinematics from Monocular Video for Surgical Robot Learning

arXiv: https://arxiv.org/abs/2512.18068 · HTML: https://mlsanity.com/paper/2512.18068/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Safe offline RL, one of 13 directions in Offline RL and imitation learning from data.
- First posted on arXiv 19 Dec 2025.

## Placed in

- Offline RL and imitation learning from data / [Safe offline RL](https://mlsanity.com/offlinerl/safe-offline-rl/)

## Built on by (most cited first, 4 of 4)

- [Real-time Rendering-based Surgical Instrument Tracking via Evolutionary Optimization](https://mlsanity.com/paper/2603.11404/) (2026)
- [SurgVIL: Scaling Surgical Robot Imitation Learning with Open-source Surgical Videos](https://mlsanity.com/paper/2608.16058/) (2026)
- [Surgical Kinematics from Monocular Video with Learned Articulated Motion Constraints](https://mlsanity.com/paper/2609.27227/) (2026)
- [Robust Surgical Robotic Instrument Tracking via Sequential Multi-Cue Fusion and Sim-to-Real Self-Training](https://mlsanity.com/paper/2610.05491/) (2026)

## Abstract (arXiv)

Imitation learning (IL) has shown immense promise in enabling autonomous dexterous manipulation, including learning surgical tasks. To fully unlock the potential of IL for surgery, access to clinical datasets is needed, which unfortunately lack the kinematic data required for current IL approaches. A promising source of large-scale surgical demonstrations is monocular surgical videos available online, making monocular pose estimation a crucial step toward enabling large-scale robot learning. Toward this end, we propose SurgiPose, a differentiable rendering based approach to estimate kinematic information from monocular surgical videos, eliminating the need for direct access to ground truth kinematics. Our method infers tool trajectories and joint angles by optimizing tool pose parameters to minimize the discrepancy between rendered and real images. To evaluate the effectiveness of our approach, we conduct experiments on two robotic surgical tasks: tissue lifting and needle pickup, using the da Vinci Research Kit Si (dVRK Si). We train imitation learning policies with both ground truth measured kinematics and estimated kinematics from video and compare their performance. Our results show that policies trained on estimated kinematics achieve comparable success rates to those trained on ground truth data, demonstrating the feasibility of using monocular video based kinematic estimation for surgical robot learning. By enabling kinematic estimation from monocular surgical videos, our work lays the foundation for large scale learning of autonomous surgical policies from online surgical data.