# Robust Box Prompt based SAM for Medical Image Segmentation

arXiv: https://arxiv.org/abs/2407.21284 · HTML: https://mlsanity.com/paper/2407.21284/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 30 references that are in the map.
- First posted on arXiv 31 Jul 2024.

## Builds on (most cited first, 18 of 18)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [FDA: Fourier Domain Adaptation for Semantic Segmentation](https://mlsanity.com/paper/2004.05498/) (2020)
- [Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentation](https://mlsanity.com/paper/2304.12620/) (2023)
- [Segment Anything in High Quality](https://mlsanity.com/paper/2306.01567/) (2023)
- [AMOS: A Large-Scale Abdominal Multi-Organ Benchmark for Versatile Medical Image Segmentation](https://mlsanity.com/paper/2206.08023/) (2022)
- [Segment Anything Model for Medical Image Analysis: an Experimental Study](https://mlsanity.com/paper/2304.10517/) (2023)
- [Deep Learning for Segmentation using an Open Large-Scale Dataset in 2D Echocardiography](https://mlsanity.com/paper/1908.06948/) (2019)
- [AbdomenCT-1K: Is Abdominal Organ Segmentation A Solved Problem?](https://mlsanity.com/paper/2010.14808/) (2020)
- [Semantic-SAM: Segment and Recognize Anything at Any Granularity](https://mlsanity.com/paper/2307.04767/) (2023)
- [Self-supervised Feature Learning via Exploiting Multi-modal Data for Retinal Disease Diagnosis](https://mlsanity.com/paper/2007.11067/) (2020)
- [Robustness of SAM: Segment Anything Under Corruptions and Beyond](https://mlsanity.com/paper/2306.07713/) (2023)
- [On the Robustness of Segment Anything](https://mlsanity.com/paper/2305.16220/) (2023)
- [An Empirical Study on the Robustness of the Segment Anything Model (SAM)](https://mlsanity.com/paper/2305.06422/) (2023)
- [Fourier Test-time Adaptation with Multi-level Consistency for Robust Classification](https://mlsanity.com/paper/2306.02544/) (2023)
- [SAM Fewshot Finetuning for Anatomical Segmentation in Medical Images](https://mlsanity.com/paper/2407.04651/) (2024)
- [Stable Segment Anything Model](https://mlsanity.com/paper/2311.15776/) (2023)

## Built on by (most cited first, 9 of 9)

- [Flip Learning: Weakly Supervised Erase to Segment Nodules in Breast Ultrasound](https://mlsanity.com/paper/2503.20685/) (2025)
- [Prompt Engineering in Segment Anything Model: Methodologies, Applications, and Emerging Challenges](https://mlsanity.com/paper/2507.09562/) (2025)
- [RP-SAM2: Refining Point Prompts for Stable Surgical Instrument Segmentation](https://mlsanity.com/paper/2504.07117/) (2025)
- [When Adaptation Hurts: Connecting Representational Drift to OOD Failures in MedSAM Fine-Tuning](https://mlsanity.com/paper/2608.21300/) (2026)
- [Prompt Mechanisms in Medical Imaging: A Comprehensive Survey](https://mlsanity.com/paper/2507.01055/) (2025)
- [Promptable Fire Segmentation: Unleashing SAM2's Potential for Real-Time Mobile Deployment with Strategic Bounding Box Guidance](https://mlsanity.com/paper/2510.21782/) (2025)
- [Rad-VLSM: A Cross-Modal Framework with Semantics-Assisted Prompting for Medical Segmentation and Diagnosis](https://mlsanity.com/paper/2605.18130/) (2026)
- [Enhancing MedSAM with a Lightweight Box Predictor for Medical Image Segmentation](https://mlsanity.com/paper/2606.04705/) (2026)
- [Cross-Modal Ultrasound-MRI Learning for Fetal Brain Ventricular Volumetry and Abnormality Screening](https://mlsanity.com/paper/2608.14763/) (2026)

## Abstract (arXiv)

The Segment Anything Model (SAM) can achieve satisfactory segmentation performance under high-quality box prompts. However, SAM's robustness is compromised by the decline in box quality, limiting its practicality in clinical reality. In this study, we propose a novel Robust Box prompt based SAM (RoBox-SAM) to ensure SAM's segmentation performance under prompts with different qualities. Our contribution is three-fold. First, we propose a prompt refinement module to implicitly perceive the potential targets, and output the offsets to directly transform the low-quality box prompt into a high-quality one. We then provide an online iterative strategy for further prompt refinement. Second, we introduce a prompt enhancement module to automatically generate point prompts to assist the box-promptable segmentation effectively. Last, we build a self-information extractor to encode the prior information from the input image. These features can optimize the image embeddings and attention calculation, thus, the robustness of SAM can be further enhanced. Extensive experiments on the large medical segmentation dataset including 99,299 images, 5 modalities, and 25 organs/targets validated the efficacy of our proposed RoBox-SAM.