# RobustMedSAM: Degradation-Resilient Medical Image Segmentation via Robust Foundation Model Adaptation

arXiv: https://arxiv.org/abs/2604.09814 · HTML: https://mlsanity.com/paper/2604.09814/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 21 references that are in the map.
- First posted on arXiv 10 Apr 2026.

## Builds on (most cited first, 16 of 16)

- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [FDA: Fourier Domain Adaptation for Semantic Segmentation](https://mlsanity.com/paper/2004.05498/) (2020)
- [EfficientSAM: Leveraged Masked Image Pretraining for Efficient Segment Anything](https://mlsanity.com/paper/2312.00863/) (2023)
- [Benchmarking the Robustness of Semantic Segmentation Models](https://mlsanity.com/paper/1908.05005/) (2019)
- [SurgicalSAM: Efficient Class Promptable Surgical Instrument Segmentation](https://mlsanity.com/paper/2308.08746/) (2023)
- [AutoSAM: Adapting SAM to Medical Images by Overloading the Prompt Encoder](https://mlsanity.com/paper/2306.06370/) (2023)
- [Unleashing the Potential of SAM for Medical Adaptation via Hierarchical Decoding](https://mlsanity.com/paper/2403.18271/) (2024)
- [One-Prompt to Segment All Medical Images](https://mlsanity.com/paper/2305.10300/) (2023)
- [RobustSAM: Segment Anything Robustly on Degraded Images](https://mlsanity.com/paper/2406.09627/) (2024)
- [Improving the Generalization of Segmentation Foundation Model under Distribution Shift via Weakly Supervised Adaptation](https://mlsanity.com/paper/2312.03502/) (2023)
- [Modality-agnostic Domain Generalizable Medical Image Segmentation by Multi-Frequency in Multi-Scale Attention](https://mlsanity.com/paper/2405.06284/) (2024)
- [Comprehensive Multimodal Segmentation in Medical Imaging: Combining YOLOv8 with SAM and HQ-SAM Models](https://mlsanity.com/paper/2310.12995/) (2023)
- [Parameter Efficient Fine-tuning via Cross Block Orchestration for Segment Anything Model](https://mlsanity.com/paper/2311.17112/) (2023)
- [Versatile Medical Image Segmentation Learned from Multi-Source Datasets via Model Self-Disambiguation](https://mlsanity.com/paper/2311.10696/) (2023)
- [Enhancing the Reliability of Segment Anything Model for Auto-Prompting Medical Image Segmentation with Uncertainty Rectification](https://mlsanity.com/paper/2311.10529/) (2023)
- [SAM-DA: Decoder Adapter for Efficient Medical Domain Adaptation](https://mlsanity.com/paper/2501.06836/) (2025)

## Built on by (most cited first, 1 of 1)

- [When Adaptation Hurts: Connecting Representational Drift to OOD Failures in MedSAM Fine-Tuning](https://mlsanity.com/paper/2608.21300/) (2026)

## Abstract (arXiv)

Medical image segmentation models built on Segment Anything Model (SAM) achieve strong performance on clean benchmarks, yet their reliability often degrades under realistic image corruptions such as noise, blur, motion artifacts, and modality-specific distortions. Existing approaches address either medical-domain adaptation or corruption robustness, but not both jointly. In SAM, we find that these capabilities are concentrated in complementary modules: the image encoder preserves medical priors, while the mask decoder governs corruption robustness. Motivated by this observation, we propose RobustMedSAM, which adopts module-wise checkpoint fusion by initializing the image encoder from MedSAM and the mask decoder from RobustSAM under a shared ViT-B architecture. We then fine-tune only the mask decoder on 35 medical datasets from MedSegBench, spanning six imaging modalities and 12 corruption types, while freezing the remaining components to preserve pretrained medical representations. We additionally investigate an SVD-based parameter-efficient variant for limited encoder adaptation. Experiments on both in-distribution and out-of-distribution benchmarks show that RobustMedSAM improves degraded-image Dice from 0.613 to 0.719 (+0.106) over SAM, demonstrating that structured fusion of complementary pretrained models is an effective and practical approach for robust medical image segmentation.