# Simultaneous Recognition and Pose Estimation of Instruments in Minimally Invasive Surgery

arXiv: https://arxiv.org/abs/1710.06668 · HTML: https://mlsanity.com/paper/1710.06668/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Oct 2017.

## Built on by (most cited first, 14 of 14)

- [TraSeTR: Track-to-Segment Transformer with Contrastive Query for Instance-level Instrument Segmentation in Robotic Surgery](https://mlsanity.com/paper/2202.08453/) (2022)
- [From Forks to Forceps: A New Framework for Instance Segmentation of Surgical Instruments](https://mlsanity.com/paper/2211.16200/) (2022)
- [SurgPose: a Dataset for Articulated Robotic Surgical Tool Pose Estimation and Tracking](https://mlsanity.com/paper/2502.11534/) (2025)
- [A Kinematic Bottleneck Approach For Pose Regression of Flexible Surgical Instruments directly from Images](https://mlsanity.com/paper/2103.00586/) (2021)
- [Surgical Scene Understanding in the Era of Foundation AI Models: A Comprehensive Review](https://mlsanity.com/paper/2502.14886/) (2025)
- [A semi-supervised Teacher-Student framework for surgical tool detection and localization](https://mlsanity.com/paper/2208.09926/) (2022)
- [A comprehensive survey on recent deep learning-based methods applied to surgical data](https://mlsanity.com/paper/2209.01435/) (2022)
- [Neural Fields for 3D Tracking of Anatomy and Surgical Instruments in Monocular Laparoscopic Video Clips](https://mlsanity.com/paper/2403.19265/) (2024)
- [Rethinking RGB-D Fusion for Semantic Segmentation in Surgical Datasets](https://mlsanity.com/paper/2407.19714/) (2024)
- [SoK: Security and Privacy Risks of Healthcare AI](https://mlsanity.com/paper/2409.07415/) (2024)
- [Enhancing Surgical Performance in Cardiothoracic Surgery with Innovations from Computer Vision and Artificial Intelligence: A Narrative Review](https://mlsanity.com/paper/2402.11288/) (2024)
- [MT3DNet: Multi-Task learning Network for 3D Surgical Scene Reconstruction](https://mlsanity.com/paper/2412.03928/) (2024)
- [Identifying Surgical Instruments in Pedagogical Cataract Surgery Videos through an Optimized Aggregation Network](https://mlsanity.com/paper/2501.02618/) (2025)
- [Real-Time Surgical Instrument Defect Detection via Non-Destructive Testing](https://mlsanity.com/paper/2510.14525/) (2025)

## Abstract (arXiv)

Detection of surgical instruments plays a key role in ensuring patient safety in minimally invasive surgery. In this paper, we present a novel method for 2D vision-based recognition and pose estimation of surgical instruments that generalizes to different surgical applications. At its core, we propose a novel scene model in order to simultaneously recognize multiple instruments as well as their parts. We use a Convolutional Neural Network architecture to embody our model and show that the cross-entropy loss is well suited to optimize its parameters which can be trained in an end-to-end fashion. An additional advantage of our approach is that instrument detection at test time is achieved while avoiding the need for scale-dependent sliding window evaluation. This allows our approach to be relatively parameter free at test time and shows good performance for both instrument detection and tracking. We show that our approach surpasses state-of-the-art results on in-vivo retinal microsurgery image data, as well as ex-vivo laparoscopic sequences.