# Deep Residual Learning for Instrument Segmentation in Robotic Surgery

arXiv: https://arxiv.org/abs/1703.08580 · HTML: https://mlsanity.com/paper/1703.08580/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Mar 2017.

## Built on by (most cited first, 22 of 22)

- [OphCLIP: Hierarchical Retrieval-Augmented Learning for Ophthalmic Surgical Video-Language Pretraining](https://mlsanity.com/paper/2411.15421/) (2024)
- [Image Compositing for Segmentation of Surgical Tools without Manual Annotations](https://mlsanity.com/paper/2102.09528/) (2021)
- [FUN-SIS: a Fully UNsupervised approach for Surgical Instrument Segmentation](https://mlsanity.com/paper/2202.08141/) (2022)
- [Methods and datasets for segmentation of minimally invasive surgical instruments in endoscopic images and videos: A review of the state of the art](https://mlsanity.com/paper/2304.13014/) (2023)
- [CaRTS: Causality-driven Robot Tool Segmentation from Vision and Kinematics Data](https://mlsanity.com/paper/2203.09475/) (2022)
- [Co-Generation and Segmentation for Generalized Surgical Instrument Segmentation on Unlabelled Data](https://mlsanity.com/paper/2103.09276/) (2021)
- [Surgical Scene Understanding in the Era of Foundation AI Models: A Comprehensive Review](https://mlsanity.com/paper/2502.14886/) (2025)
- [Rethinking Causality-driven Robot Tool Segmentation with Temporal Constraints](https://mlsanity.com/paper/2212.00072/) (2022)
- [ReconResNet: Regularised Residual Learning for MR Image Reconstruction of Undersampled Cartesian and Radial Data](https://mlsanity.com/paper/2103.09203/) (2021)
- [Histogram of Oriented Gradients Meet Deep Learning: A Novel Multi-task Deep Network for Medical Image Semantic Segmentation](https://mlsanity.com/paper/2204.01712/) (2022)
- [Exploring Deep Learning Methods for Real-Time Surgical Instrument Segmentation in Laparoscopy](https://mlsanity.com/paper/2107.02319/) (2021)
- [A comprehensive survey on recent deep learning-based methods applied to surgical data](https://mlsanity.com/paper/2209.01435/) (2022)
- [Revisiting Surgical Instrument Segmentation Without Human Intervention: A Graph Partitioning View](https://mlsanity.com/paper/2408.14789/) (2024)
- [AMNCutter: Affinity-Attention-Guided Multi-View Normalized Cutter for Unsupervised Surgical Instrument Segmentation](https://mlsanity.com/paper/2411.03695/) (2024)
- [Parsing Objects at a Finer Granularity: A Survey](https://mlsanity.com/paper/2212.13693/) (2022)
- [SAF-IS: a Spatial Annotation Free Framework for Instance Segmentation of Surgical Tools](https://mlsanity.com/paper/2309.01723/) (2023)
- [Visual-Kinematics Graph Learning for Procedure-agnostic Instrument Tip Segmentation in Robotic Surgeries](https://mlsanity.com/paper/2309.00957/) (2023)
- [Identifying Surgical Instruments in Laparoscopy Using Deep Learning Instance Segmentation](https://mlsanity.com/paper/2508.21399/) (2025)
- [Segmenting Medical Instruments in Minimally Invasive Surgeries using AttentionMask](https://mlsanity.com/paper/2203.11358/) (2022)
- [LapSeg3D: Weakly Supervised Semantic Segmentation of Point Clouds Representing Laparoscopic Scenes](https://mlsanity.com/paper/2207.07418/) (2022)
- [Plug-in for visualizing 3D tool tracking from videos of Minimally Invasive Surgeries](https://mlsanity.com/paper/2401.09472/) (2024)
- [MT3DNet: Multi-Task learning Network for 3D Surgical Scene Reconstruction](https://mlsanity.com/paper/2412.03928/) (2024)

## Abstract (arXiv)

Detection, tracking, and pose estimation of surgical instruments are crucial tasks for computer assistance during minimally invasive robotic surgery. In the majority of cases, the first step is the automatic segmentation of surgical tools. Prior work has focused on binary segmentation, where the objective is to label every pixel in an image as tool or background. We improve upon previous work in two major ways. First, we leverage recent techniques such as deep residual learning and dilated convolutions to advance binary-segmentation performance. Second, we extend the approach to multi-class segmentation, which lets us segment different parts of the tool, in addition to background. We demonstrate the performance of this method on the MICCAI Endoscopic Vision Challenge Robotic Instruments dataset.