# U-NetPlus: A Modified Encoder-Decoder U-Net Architecture for Semantic and Instance Segmentation of Surgical Instrument

arXiv: https://arxiv.org/abs/1902.08994 · HTML: https://mlsanity.com/paper/1902.08994/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Feb 2019.

## Built on by (most cited first, 23 of 23)

- [MATIS: Masked-Attention Transformers for Surgical Instrument Segmentation](https://mlsanity.com/paper/2303.09514/) (2023)
- [From Forks to Forceps: A New Framework for Instance Segmentation of Surgical Instruments](https://mlsanity.com/paper/2211.16200/) (2022)
- [FUN-SIS: a Fully UNsupervised approach for Surgical Instrument Segmentation](https://mlsanity.com/paper/2202.08141/) (2022)
- [Methods and datasets for segmentation of minimally invasive surgical instruments in endoscopic images and videos: A review of the state of the art](https://mlsanity.com/paper/2304.13014/) (2023)
- [Deep Learning for Surgical Instrument Recognition and Segmentation in Robotic-Assisted Surgeries: A Systematic Review](https://mlsanity.com/paper/2410.07269/) (2024)
- [SegCol Challenge: Semantic Segmentation for Tools and Fold Edges in Colonoscopy data](https://mlsanity.com/paper/2412.16078/) (2024)
- [How can we learn (more) from challenges? A statistical approach to driving future algorithm development](https://mlsanity.com/paper/2106.09302/) (2021)
- [SegMatch: A semi-supervised learning method for surgical instrument segmentation](https://mlsanity.com/paper/2308.05232/) (2023)
- [A comprehensive survey on recent deep learning-based methods applied to surgical data](https://mlsanity.com/paper/2209.01435/) (2022)
- [TTT-Unet: Enhancing U-Net with Test-Time Training Layers for Biomedical Image Segmentation](https://mlsanity.com/paper/2409.11299/) (2024)
- [SAF-IS: a Spatial Annotation Free Framework for Instance Segmentation of Surgical Tools](https://mlsanity.com/paper/2309.01723/) (2023)
- [Rethinking RGB-D Fusion for Semantic Segmentation in Surgical Datasets](https://mlsanity.com/paper/2407.19714/) (2024)
- [Spatial-Temporal Map Vehicle Trajectory Detection Using Dynamic Mode Decomposition and Res-UNet+ Neural Networks](https://mlsanity.com/paper/2201.04755/) (2022)
- [Deep Instance Segmentation and Visual Servoing to Play Jenga with a Cost-Effective Robotic System](https://mlsanity.com/paper/2211.07977/) (2022)
- [Identifying Surgical Instruments in Laparoscopy Using Deep Learning Instance Segmentation](https://mlsanity.com/paper/2508.21399/) (2025)
- [Crack Semantic Segmentation using the U-Net with Full Attention Strategy](https://mlsanity.com/paper/2104.14586/) (2021)
- [LapSeg3D: Weakly Supervised Semantic Segmentation of Point Clouds Representing Laparoscopic Scenes](https://mlsanity.com/paper/2207.07418/) (2022)
- [Evaluation of the potential of Near Infrared Hyperspectral Imaging for monitoring the invasive brown marmorated stink bug](https://mlsanity.com/paper/2301.08252/) (2023)
- [Label-supervised surgical instrument segmentation using temporal equivariance and semantic continuity](https://mlsanity.com/paper/2403.09551/) (2024)
- [LACOSTE: Exploiting stereo and temporal contexts for surgical instrument segmentation](https://mlsanity.com/paper/2409.09360/) (2024)
- [Convolutional Neural Network Segmentation for Satellite Imagery Data to Identify Landforms Using U-Net Architecture](https://mlsanity.com/paper/2502.05476/) (2025)
- [NeeCo: Image Synthesis of Novel Instrument States Based on Dynamic and Deformable 3D Gaussian Reconstruction](https://mlsanity.com/paper/2508.07897/) (2025)
- [Toward Mask Annotation-Free Surgical Instrument Segmentation from Endoscopic Images Using Text-Prompted Segment Anything Model 3 (SAM3)](https://mlsanity.com/paper/2608.08844/) (2026)

## Abstract (arXiv)

Conventional therapy approaches limit surgeons' dexterity control due to limited field-of-view. With the advent of robot-assisted surgery, there has been a paradigm shift in medical technology for minimally invasive surgery. However, it is very challenging to track the position of the surgical instruments in a surgical scene, and accurate detection & identification of surgical tools is paramount. Deep learning-based semantic segmentation in frames of surgery videos has the potential to facilitate this task. In this work, we modify the U-Net architecture named U-NetPlus, by introducing a pre-trained encoder and re-design the decoder part, by replacing the transposed convolution operation with an upsampling operation based on nearest-neighbor (NN) interpolation. To further improve performance, we also employ a very fast and flexible data augmentation technique. We trained the framework on 8 x 225 frame sequences of robotic surgical videos, available through the MICCAI 2017 EndoVis Challenge dataset and tested it on 8 x 75 frame and 2 x 300 frame videos. Using our U-NetPlus architecture, we report a 90.20% DICE for binary segmentation, 76.26% DICE for instrument part segmentation, and 46.07% for instrument type (i.e., all instruments) segmentation, outperforming the results of previous techniques implemented and tested on these data.