# SIRA: Reasoning-Aware Surgical Instrument Segmentation via Query-Anchored Alignment

arXiv: https://arxiv.org/abs/2609.21402 · HTML: https://mlsanity.com/paper/2609.21402/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 23 references that are in the map.
- First posted on arXiv 18 Sep 2026.

## Builds on (most cited first, 22 of 22)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [LISA: Reasoning Segmentation via Large Language Model](https://mlsanity.com/paper/2308.00692/) (2023)
- [GLaMM: Pixel Grounding Large Multimodal Model](https://mlsanity.com/paper/2311.03356/) (2023)
- [Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understanding](https://mlsanity.com/paper/2311.08046/) (2023)
- [Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcement](https://mlsanity.com/paper/2503.06520/) (2025)
- [2017 Robotic Instrument Segmentation Challenge](https://mlsanity.com/paper/1902.06426/) (2019)
- [GSVA: Generalized Segmentation via Multimodal Large Language Models](https://mlsanity.com/paper/2312.10103/) (2023)
- [SurgicalSAM: Efficient Class Promptable Surgical Instrument Segmentation](https://mlsanity.com/paper/2308.08746/) (2023)
- [SAR-RARP50: Segmentation of surgical instrumentation and Action Recognition on Robot-Assisted Radical Prostatectomy Challenge](https://mlsanity.com/paper/2401.00496/) (2023)
- [MATIS: Masked-Attention Transformers for Surgical Instrument Segmentation](https://mlsanity.com/paper/2303.09514/) (2023)
- [Transforming medical imaging with Transformers? A comparative review of key properties, current progresses, and future perspectives](https://mlsanity.com/paper/2206.01136/) (2022)
- [SurgVLM: A Large Vision-Language Model and Systematic Evaluation Benchmark for Surgical Intelligence](https://mlsanity.com/paper/2506.02555/) (2025)
- [Text Promptable Surgical Instrument Segmentation with Vision-Language Models](https://mlsanity.com/paper/2306.09244/) (2023)
- [The Devil is in Temporal Token: High Quality Video Reasoning Segmentation](https://mlsanity.com/paper/2501.08549/) (2025)
- [Surgical Visual Understanding (SurgVU) Dataset](https://mlsanity.com/paper/2501.09209/) (2025)
- [Video Dataset for Surgical Phase, Keypoint, and Instrument Recognition in Laparoscopic Surgery (PhaKIR)](https://mlsanity.com/paper/2511.06549/) (2025)
- [SurgicalPart-SAM: Part-to-Whole Collaborative Prompting for Surgical Instrument Segmentation](https://mlsanity.com/paper/2312.14481/) (2023)
- [PathMR: Multimodal Visual Reasoning for Interpretable Pathology Diagnosis](https://mlsanity.com/paper/2508.20851/) (2025)

## Abstract (arXiv)

Surgical instrument segmentation (SIS) plays a critical role in robotic assistance and surgical workflow analysis. However, most existing SIS methods formulate segmentation as a category-driven localization problem, limiting their ability to capture procedural context and task-dependent semantics in surgical workflows. We introduce Reasoning-Aware Surgical Instrument Segmentation (RA-SIS), a task formulation that frames segmentation as query-conditioned inference under surgical context. To benchmark this setting, we construct SurgRS, a surgical reasoning segmentation dataset consisting of 41,000 image-text pairs, which aligns instance-level masks with structured query-answer supervision to enable semantic grounding at the pixel level. Based on SurgRS, we propose Surgical Instrument Reasoning and Segmentation Assistant (SIRA), a multimodal framework that disentangles target-level and query-level semantics and integrates them with visual features through query-anchored dual alignment. By aligning query semantics with spatial features and segmentation prompts, SIRA enhances semantic-visual consistency in mask prediction. Extensive experiments on SurgRS demonstrate improvements over existing reasoning-aware baselines. Code is available at https://github.com/linxir226/SIRA.