SIRA: Reasoning-Aware Surgical Instrument Segmentation via Query-Anchored Alignment
Not on the map. Why?
Abstract (arXiv)
Surgical instrument segmentation (SIS) plays a critical role in robotic assistance and surgical workflow analysis. However, most existing SIS methods formulate segmentation as a category-driven localization problem, limiting their ability to capture procedural context and task-dependent semantics in surgical workflows. We introduce Reasoning-Aware Surgical Instrument Segmentation (RA-SIS), a task formulation that frames segmentation as query-conditioned inference under surgical context. To benchmark this setting, we construct SurgRS, a surgical reasoning segmentation dataset consisting of 41,000 image-text pairs, which aligns instance-level masks with structured query-answer supervision to enable semantic grounding at the pixel level. Based on SurgRS, we propose Surgical Instrument Reasoning and Segmentation Assistant (SIRA), a multimodal framework that disentangles target-level and query-level semantics and integrates them with visual features through query-anchored dual alignment. By aligning query semantics with spatial features and segmentation prompts, SIRA enhances semantic-visual consistency in mask prediction. Extensive experiments on SurgRS demonstrate improvements over existing reasoning-aware baselines. Code is available at https://github.com/linxir226/SIRA.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 22 of its 23 references that are in the map. First posted on arXiv 18 Sep 2026.
Builds on
22 of 23 references.
References · 22
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
- 2023Segment Anythingcited by 8,234
- 2017Mask R-CNNcited by 7,277
- 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
- 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
- 2023LISA: Reasoning Segmentation via Large Language Modelcited by 776
- 2023GLaMM: Pixel Grounding Large Multimodal Modelcited by 455
- 2023Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understandingcited by 371
- 2025Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcementcited by 239
- 20192017 Robotic Instrument Segmentation Challengecited by 168
- 2023GSVA: Generalized Segmentation via Multimodal Large Language Modelscited by 154
- 2023SurgicalSAM: Efficient Class Promptable Surgical Instrument Segmentationcited by 104
10 more
- 2023SAR-RARP50: Segmentation of surgical instrumentation and Action Recognition on Robot-Assisted Radical Prostatectomy Challengecited by 56
- 2023MATIS: Masked-Attention Transformers for Surgical Instrument Segmentationcited by 36
- 2022Transforming medical imaging with Transformers? A comparative review of key properties, current progresses, and future perspectivescited by 33
- 2025SurgVLM: A Large Vision-Language Model and Systematic Evaluation Benchmark for Surgical Intelligencecited by 31
- 2023Text Promptable Surgical Instrument Segmentation with Vision-Language Modelscited by 28
- 2025The Devil is in Temporal Token: High Quality Video Reasoning Segmentationcited by 26
- 2025Surgical Visual Understanding (SurgVU) Datasetcited by 11
- 2025Video Dataset for Surgical Phase, Keypoint, and Instrument Recognition in Laparoscopic Surgery (PhaKIR)cited by 9
- 2023SurgicalPart-SAM: Part-to-Whole Collaborative Prompting for Surgical Instrument Segmentationcited by 8
- 2025PathMR: Multimodal Visual Reasoning for Interpretable Pathology Diagnosiscited by 2
Built on by
Not cited yet.