mlsanityData to 7 Oct 2026

MATIS: Masked-Attention Transformers for Surgical Instrument Segmentation

arXiv 2303.09514 ↗First posted 16 Mar 2023cs.CVcited by 36

Not on the map. Why?

Abstract (arXiv)

We propose Masked-Attention Transformers for Surgical Instrument Segmentation (MATIS), a two-stage, fully transformer-based method that leverages modern pixel-wise attention mechanisms for instrument segmentation. MATIS exploits the instance-level nature of the task by employing a masked attention module that generates and classifies a set of fine instrument region proposals. Our method incorporates long-term video-level information through video transformers to improve temporal consistency and enhance mask classification. We validate our approach in the two standard public benchmarks, Endovis 2017 and Endovis 2018. Our experiments demonstrate that MATIS' per-frame baseline outperforms previous state-of-the-art methods and that including our temporal consistency module boosts our model's performance further.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). Builds on 28 of its 38 references that are in the map. First posted on arXiv 16 Mar 2023.

Builds on

28 of 38 references.

References · 28

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  4. 2020End-to-End Object Detection with Transformerscited by 7,384
  5. 2017Mask R-CNNcited by 7,259
  6. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  7. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
  8. 2017The Kinetics Human Action Video Datasetcited by 1,894
  9. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,148
  10. 2021Video Swin Transformercited by 1,090
  11. 2021Multiscale Vision Transformerscited by 896
  12. 20192017 Robotic Instrument Segmentation Challengecited by 166
16 more
  1. 20202018 Robotic Scene Segmentation Challengecited by 123
  2. 2018Automatic Instrument Segmentation in Robot-Assisted Surgery Using Deep Learningcited by 81
  3. 2020ISINet: An Instance-Based Approach for Surgical Instrument Segmentationcited by 67
  4. 2019Incorporating Temporal Prior from Motion Flow for Instrument Segmentation in Minimally Invasive Surgery Videocited by 64
  5. 2022TraSeTR: Track-to-Segment Transformer with Contrastive Query for Instance-level Instrument Segmentation in Robotic Surgerycited by 40
  6. 2018Comparative evaluation of instrument segmentation and tracking methods in minimally invasive surgerycited by 36
  7. 2020Learning Motion Flows for Semi-supervised Instrument Segmentation from Robotic Surgical Videocited by 35
  8. 2022Towards Holistic Surgical Scene Understandingcited by 33
  9. 2017ToolNet: Holistically-Nested Real-Time Segmentation of Robotic Surgical Toolscited by 30
  10. 2022Exploring Intra- and Inter-Video Relation for Surgical Semantic Scene Segmentationcited by 27
  11. 2020Synthetic and Real Inputs for Tool Segmentation in Robotic Surgerycited by 25
  12. 2017Concurrent Segmentation and Localization for Tracking of Surgical Instrumentscited by 24
  13. 2019U-NetPlus: A Modified Encoder-Decoder U-Net Architecture for Semantic and Instance Segmentation of Surgical Instrumentcited by 23
  14. 2019Learning Where to Look While Tracking Instruments in Robot-assisted Surgerycited by 17
  15. 2019Self-Supervised Surgical Tool Segmentation using Kinematic Informationcited by 15
  16. 2021One to Many: Adaptive Instrument Segmentation via Meta Learning and Dynamic Online Adaptation in Robotic Surgical Videocited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 10; 2025: 13; 2026: 9.

36 papers.

Sort:

By area:

  1. 2023SurgicalSAM: Efficient Class Promptable Surgical Instrument Segmentationcited by 103
  2. 2023SAR-RARP50: Segmentation of surgical instrumentation and Action Recognition on Robot-Assisted Radical Prostatectomy Challengecited by 54
  3. 2024Pixel-Wise Recognition for Holistic Surgical Scene Understandingcited by 48
  4. 2024Surgical SAM 2: Real-time Segment Anything in Surgical Video by Efficient Frame Pruningcited by 46
  5. 2023Text Promptable Surgical Instrument Segmentation with Vision-Language Modelscited by 27
  6. 2023SurgicalPart-SAM: Part-to-Whole Collaborative Prompting for Surgical Instrument Segmentationcited by 8
  7. 2025Memory-Augmented SAM2 for Training-Free Surgical Video Segmentationcited by 8
  8. 2025Comparative validation of surgical phase recognition, instrument keypoint estimation, and instrument instance segmentation in endoscopy: Results of the PhaKIR 2024 challengecited by 6
  9. 2024Segment Anything for Videos: A Systematic Surveycited by 5
  10. 2024Tracking Everything in Robotic-Assisted Surgerycited by 5
  11. 2025ProstaTD: Bridging Surgical Triplet from Classification to Fully Supervised Detectioncited by 4
  12. 2024Phase-Informed Tool Segmentation for Manual Small-Incision Cataract Surgerycited by 3
24 more
  1. 2025Surgical Action Planning with Large Language Modelscited by 3
  2. 2025Cataract-LMM Large-Scale Multi-Source Multi-Task Benchmark for Deep Learning in Surgical Video Analysiscited by 3
  3. 2026Where It Moves, It Matters: Referring Surgical Instrument Segmentation via Motioncited by 3
  4. 2026From Articulated Kinematics to Routed Visual Control for Action-Conditioned Surgical Video Generationcited by 3
  5. 2024Personalizing Federated Instrument Segmentation with Visual Trait Priors in Robotic Surgerycited by 2
  6. 2025Recognize Any Surgical Object: Unleashing the Power of Weakly-Supervised Datacited by 2
  7. 2024Image Synthesis with Class-Aware Semantic Diffusion Models for Surgical Scene Segmentationcited by 1
  8. 2025TSMS-SAM2: Multi-scale Temporal Sampling Augmentation and Memory-Splitting Pruning for Promptable Video Object Segmentation and Tracking in Surgical Scenarioscited by 1
  9. 2025SurgSLOT: Segment Anything in Surgical Videos via Semantic Long-term Trackingcited by 1
  10. 2026An Approach to Enriching Surgical Video Datasets for Fine-Grained Spatial-Temporal Understanding of Vision-Language Modelscited by 1
  11. 2026UniSurgSAM: A Unified Promptable Model for Reliable Surgical Video Segmentationcited by 1
  12. 2026Hierarchical Prototype-Memory Adaptation of SAM for Surgical Instrument Segmentationcited by 1
  13. 2024Label-supervised surgical instrument segmentation using temporal equivariance and semantic continuitycited by 0
  14. 2024LACOSTE: Exploiting stereo and temporal contexts for surgical instrument segmentationcited by 0
  15. 2024Rethinking Text-Promptable Surgical Instrument Segmentation with Robust Frameworkcited by 0
  16. 2025CLIP-RL: Surgical Scene Segmentation Using Contrastive Language-Vision Pretraining & Reinforcement Learningcited by 0
  17. 2025SurgPIS: Surgical-instrument-level Instances and Part-level Semantics for Weakly-supervised Part-aware Instance Segmentationcited by 0
  18. 2025FASL-Seg: Anatomy and Tool Segmentation of Surgical Scenescited by 0
  19. 2025ReMeDI: Refined Memory for Disambiguation of Identities with SAM3 in Surgical Segmentationcited by 0
  20. 2025Toward Real-Time Surgical Scene Segmentation via a Spike-Driven Video Transformer with Spike-Informed Pretrainingcited by 0
  21. 2026Toward Mask Annotation-Free Surgical Instrument Segmentation from Endoscopic Images Using Text-Prompted Segment Anything Model 3 (SAM3)cited by 0
  22. 2026ReGround-Surg: Reliability-Guided Anchor Grounding for Referring Surgical Video Segmentationcited by 0
  23. 2026TEDi: Temporal Memory-Enhanced and Denoising Transformer for Surgical Instrument Segmentationcited by 0
  24. 2026SIRA: Reasoning-Aware Surgical Instrument Segmentation via Query-Anchored Alignmentcited by 0