# AMNCutter: Affinity-Attention-Guided Multi-View Normalized Cutter for Unsupervised Surgical Instrument Segmentation

arXiv: https://arxiv.org/abs/2411.03695 · HTML: https://mlsanity.com/paper/2411.03695/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 66 references that are in the map.
- First posted on arXiv 6 Nov 2024.

## Builds on (most cited first, 30 of 43)

- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Benchmarking Graph Neural Networks](https://mlsanity.com/paper/2003.00982/) (2020)
- [An Efficient Graph Convolutional Network Technique for the Travelling Salesman Problem](https://mlsanity.com/paper/1906.01227/) (2019)
- [Unsupervised Semantic Segmentation by Distilling Feature Correspondences](https://mlsanity.com/paper/2203.08414/) (2022)
- [Localizing Objects with Self-Supervised Transformers and no Labels](https://mlsanity.com/paper/2109.14279/) (2021)
- [Cut and Learn for Unsupervised Object Detection and Instance Segmentation](https://mlsanity.com/paper/2301.11320/) (2023)
- [Deep Spectral Methods: A Surprisingly Strong Baseline for Unsupervised Semantic Segmentation and Localization](https://mlsanity.com/paper/2205.07839/) (2022)
- [Surgical Data Science: Enabling Next-Generation Surgery](https://mlsanity.com/paper/1701.06482/) (2017)
- [Normalized Cut Loss for Weakly-supervised CNN Segmentation](https://mlsanity.com/paper/1804.01346/) (2018)
- [CholecSeg8k: A Semantic Segmentation Dataset for Laparoscopic Cholecystectomy Based on Cholec80](https://mlsanity.com/paper/2012.12453/) (2020)
- [Surgical Data Science -- from Concepts toward Clinical Translation](https://mlsanity.com/paper/2011.02284/) (2020)
- [SurgicalSAM: Efficient Class Promptable Surgical Instrument Segmentation](https://mlsanity.com/paper/2308.08746/) (2023)
- [Automatic Instrument Segmentation in Robot-Assisted Surgery Using Deep Learning](https://mlsanity.com/paper/1803.01207/) (2018)
- [ISINet: An Instance-Based Approach for Surgical Instrument Segmentation](https://mlsanity.com/paper/2007.05533/) (2020)
- [Incorporating Temporal Prior from Motion Flow for Instrument Segmentation in Minimally Invasive Surgery Video](https://mlsanity.com/paper/1907.07899/) (2019)
- [Unsupervised Learning of Image Segmentation Based on Differentiable Feature Clustering](https://mlsanity.com/paper/2007.09990/) (2020)
- [ACSeg: Adaptive Conceptualization for Unsupervised Semantic Segmentation](https://mlsanity.com/paper/2210.05944/) (2022)
- [Unsupervised Instance Segmentation in Microscopy Images via Panoptic Domain Adaptation and Task Re-weighting](https://mlsanity.com/paper/2005.02066/) (2020)
- [AdaptiveSAM: Towards Efficient Tuning of SAM for Surgical Scene Segmentation](https://mlsanity.com/paper/2308.03726/) (2023)
- [ToolNet: Holistically-Nested Real-Time Segmentation of Robotic Surgical Tools](https://mlsanity.com/paper/1706.08126/) (2017)
- [Synthetic and Real Inputs for Tool Segmentation in Robotic Surgery](https://mlsanity.com/paper/2007.09107/) (2020)
- [DeepCut: Unsupervised Segmentation using Graph Neural Networks Clustering](https://mlsanity.com/paper/2212.05853/) (2022)
- [Concurrent Segmentation and Localization for Tracking of Surgical Instruments](https://mlsanity.com/paper/1703.10701/) (2017)
- [Deep Residual Learning for Instrument Segmentation in Robotic Surgery](https://mlsanity.com/paper/1703.08580/) (2017)
- [Adversarial Attack on Community Detection by Hiding Individuals](https://mlsanity.com/paper/2001.07933/) (2020)
- [FUN-SIS: a Fully UNsupervised approach for Surgical Instrument Segmentation](https://mlsanity.com/paper/2202.08141/) (2022)
- [Learning Where to Look While Tracking Instruments in Robot-assisted Surgery](https://mlsanity.com/paper/1907.00214/) (2019)
- [Real-time Instance Segmentation of Surgical Instruments using Attention and Multi-scale Feature Fusion](https://mlsanity.com/paper/2111.04911/) (2021)
- [Self-Supervised Surgical Tool Segmentation using Kinematic Information](https://mlsanity.com/paper/1902.04810/) (2019)

## Built on by (most cited first, 3 of 3)

- [Surgical Scene Understanding in the Era of Foundation AI Models: A Comprehensive Review](https://mlsanity.com/paper/2502.14886/) (2025)
- [SurgiATM: A Physics-Guided Plug-and-Play Model for Deep Learning-Based Smoke Removal in Laparoscopic Surgery](https://mlsanity.com/paper/2511.05059/) (2025)
- [PhySe-RPO: Physics and Semantics Guided Relative Policy Optimization for Diffusion-Based Surgical Smoke Removal](https://mlsanity.com/paper/2603.22844/) (2026)

## Abstract (arXiv)

Surgical instrument segmentation (SIS) is pivotal for robotic-assisted minimally invasive surgery, assisting surgeons by identifying surgical instruments in endoscopic video frames. Recent unsupervised surgical instrument segmentation (USIS) methods primarily rely on pseudo-labels derived from low-level features such as color and optical flow, but these methods show limited effectiveness and generalizability in complex and unseen endoscopic scenarios. In this work, we propose a label-free unsupervised model featuring a novel module named Multi-View Normalized Cutter (m-NCutter). Different from previous USIS works, our model is trained using a graph-cutting loss function that leverages patch affinities for supervision, eliminating the need for pseudo-labels. The framework adaptively determines which affinities from which levels should be prioritized. Therefore, the low- and high-level features and their affinities are effectively integrated to train a label-free unsupervised model, showing superior effectiveness and generalization ability. We conduct comprehensive experiments across multiple SIS datasets to validate our approach's state-of-the-art (SOTA) performance, robustness, and exceptional potential as a pre-trained model. Our code is released at https://github.com/MingyuShengSMY/AMNCutter.