# Adaptive Interactive Segmentation for Multimodal Medical Imaging via Selection Engine

arXiv: https://arxiv.org/abs/2411.19447 · HTML: https://mlsanity.com/paper/2411.19447/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 38 references that are in the map.
- First posted on arXiv 29 Nov 2024.

## Builds on (most cited first, 25 of 25)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [UNet++: A Nested U-Net Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1807.10165/) (2018)
- [Kvasir-SEG: A Segmented Polyp Dataset](https://mlsanity.com/paper/1911.07069/) (2019)
- [Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentation](https://mlsanity.com/paper/2304.12620/) (2023)
- [Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Models](https://mlsanity.com/paper/2402.17177/) (2024)
- [AMOS: A Large-Scale Abdominal Multi-Organ Benchmark for Versatile Medical Image Segmentation](https://mlsanity.com/paper/2206.08023/) (2022)
- [nnU-Net: Self-adapting Framework for U-Net-Based Medical Image Segmentation](https://mlsanity.com/paper/1809.10486/) (2018)
- [ResUNet-a: a deep learning framework for semantic segmentation of remotely sensed data](https://mlsanity.com/paper/1904.00592/) (2019)
- [Segment Anything Model (SAM) for Digital Pathology: Assess Zero-shot Segmentation on Whole Slide Imaging](https://mlsanity.com/paper/2304.04155/) (2023)
- [Medical SAM 2: Segment medical images as video via Segment Anything Model 2](https://mlsanity.com/paper/2408.00874/) (2024)
- [Interactive Medical Image Segmentation using Deep Learning with Image-specific Fine-tuning](https://mlsanity.com/paper/1710.04043/) (2017)
- [DeepIGeoS: A Deep Interactive Geodesic Framework for Medical Image Segmentation](https://mlsanity.com/paper/1707.00652/) (2017)
- [SAM on Medical Images: A Comprehensive Study on Three Prompt Modes](https://mlsanity.com/paper/2305.00035/) (2023)
- [SAM.MD: Zero-shot medical image segmentation capabilities of the Segment Anything Model](https://mlsanity.com/paper/2304.05396/) (2023)
- [When SAM Meets Medical Images: An Investigation of Segment Anything Model (SAM) on Multi-phase Liver Tumor Segmentation](https://mlsanity.com/paper/2304.08506/) (2023)
- [Abdominal multi-organ segmentation with organ-attention networks and statistical fusion](https://mlsanity.com/paper/1804.08414/) (2018)
- [MIDeepSeg: Minimally Interactive Segmentation of Unseen Objects from Medical Images Using Deep Learning](https://mlsanity.com/paper/2104.12166/) (2021)
- [Segment Anything in Medical Images and Videos: Benchmark and Deployment](https://mlsanity.com/paper/2408.03322/) (2024)
- [One-Prompt to Segment All Medical Images](https://mlsanity.com/paper/2305.10300/) (2023)
- [CheXmask: a large-scale dataset of anatomical segmentation masks for multi-center chest x-ray images](https://mlsanity.com/paper/2307.03293/) (2023)
- [m2caiSeg: Semantic Segmentation of Laparoscopic Images using Convolutional Neural Networks](https://mlsanity.com/paper/2008.10134/) (2020)
- [Ladder Fine-tuning approach for SAM integrating complementary network](https://mlsanity.com/paper/2306.12737/) (2023)
- [Deep-Learning Ensembles for Skin-Lesion Segmentation, Analysis, Classification: RECOD Titans at ISIC Challenge 2018](https://mlsanity.com/paper/1808.08480/) (2018)

## Built on by (most cited first, 1 of 1)

- [Implicit Shape-Prior for Few-Shot Assisted 3D Segmentation](https://mlsanity.com/paper/2509.08580/) (2025)

## Abstract (arXiv)

In medical image analysis, achieving fast, efficient, and accurate segmentation is essential for automated diagnosis and treatment. Although recent advancements in deep learning have significantly improved segmentation accuracy, current models often face challenges in adaptability and generalization, particularly when processing multi-modal medical imaging data. These limitations stem from the substantial variations between imaging modalities and the inherent complexity of medical data. To address these challenges, we propose the Strategy-driven Interactive Segmentation Model (SISeg), built on SAM2, which enhances segmentation performance across various medical imaging modalities by integrating a selection engine. To mitigate memory bottlenecks and optimize prompt frame selection during the inference of 2D image sequences, we developed an automated system, the Adaptive Frame Selection Engine (AFSE). This system dynamically selects the optimal prompt frames without requiring extensive prior medical knowledge and enhances the interpretability of the model's inference process through an interactive feedback mechanism. We conducted extensive experiments on 10 datasets covering 7 representative medical imaging modalities, demonstrating the SISeg model's robust adaptability and generalization in multi-modal tasks. The project page and code will be available at: [URL].