# SAM.MD: Zero-shot medical image segmentation capabilities of the Segment Anything Model

arXiv: https://arxiv.org/abs/2304.05396 · HTML: https://mlsanity.com/paper/2304.05396/

## Key facts

- Cited by 61 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 9 references that are in the map.
- First posted on arXiv 10 Apr 2023.

## Builds on (most cited first, 8 of 8)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Image Segmentation Using Text and Image Prompts](https://mlsanity.com/paper/2112.10003/) (2021)
- [AMOS: A Large-Scale Abdominal Multi-Organ Benchmark for Versatile Medical Image Segmentation](https://mlsanity.com/paper/2206.08023/) (2022)
- [Generalized Decoding for Pixel, Image, and Language](https://mlsanity.com/paper/2212.11270/) (2022)
- [nnU-Net: Self-adapting Framework for U-Net-Based Medical Image Segmentation](https://mlsanity.com/paper/1809.10486/) (2018)

## Built on by (most cited first, 30 of 61)

- [Multimodal Foundation Models: From Specialists to General-Purpose Assistants](https://mlsanity.com/paper/2309.10020/) (2023)
- [SurgicalSAM: Efficient Class Promptable Surgical Instrument Segmentation](https://mlsanity.com/paper/2308.08746/) (2023)
- [SAM on Medical Images: A Comprehensive Study on Three Prompt Modes](https://mlsanity.com/paper/2305.00035/) (2023)
- [MA-SAM: Modality-agnostic SAM Adaptation for 3D Medical Image Segmentation](https://mlsanity.com/paper/2309.08842/) (2023)
- [Review of Large Vision Models and Visual Prompt Engineering](https://mlsanity.com/paper/2307.00855/) (2023)
- [Polyp-SAM: Transfer SAM for Polyp Segmentation](https://mlsanity.com/paper/2305.00293/) (2023)
- [A Comprehensive Survey on Segment Anything Model for Vision and Beyond](https://mlsanity.com/paper/2305.08196/) (2023)
- [SAM2-Adapter: Evaluating & Adapting Segment Anything 2 in Downstream Tasks: Camouflage, Shadow, Medical Image Segmentation, and More](https://mlsanity.com/paper/2408.04579/) (2024)
- [Can SAM Segment Polyps?](https://mlsanity.com/paper/2304.07583/) (2023)
- [Unleashing the Potential of SAM for Medical Adaptation via Hierarchical Decoding](https://mlsanity.com/paper/2403.18271/) (2024)
- [A Survey on Segment Anything Model (SAM): Vision Foundation Model Meets Prompt Engineering](https://mlsanity.com/paper/2306.06211/) (2023)
- [SAMAug: Point Prompt Augmentation for Segment Anything Model](https://mlsanity.com/paper/2307.01187/) (2023)
- [Image Segmentation in Foundation Model Era: A Survey](https://mlsanity.com/paper/2408.12957/) (2024)
- [Artificial General Intelligence for Medical Imaging Analysis](https://mlsanity.com/paper/2306.05480/) (2023)
- [Foundation Models for Biomedical Image Segmentation: A Survey](https://mlsanity.com/paper/2401.07654/) (2024)
- [Deep Interactive Segmentation of Medical Images: A Systematic Review and Taxonomy](https://mlsanity.com/paper/2311.13964/) (2023)
- [DeSAM: Decoupled Segment Anything Model for Generalizable Medical Image Segmentation](https://mlsanity.com/paper/2306.00499/) (2023)
- [APSeg: Auto-Prompt Network for Cross-Domain Few-Shot Semantic Segmentation](https://mlsanity.com/paper/2406.08372/) (2024)
- [On the Robustness of Segment Anything](https://mlsanity.com/paper/2305.16220/) (2023)
- [Polyp-SAM++: Can A Text Guided SAM Perform Better for Polyp Segmentation?](https://mlsanity.com/paper/2308.06623/) (2023)
- [Towards Segment Anything Model (SAM) for Medical Image Segmentation: A Survey](https://mlsanity.com/paper/2305.03678/) (2023)
- [Self-Sampling Meta SAM: Enhancing Few-shot Medical Image Segmentation with Meta-Learning](https://mlsanity.com/paper/2308.16466/) (2023)
- [All in One Framework for Multimodal Re-identification in the Wild](https://mlsanity.com/paper/2405.04741/) (2024)
- [SAM-DA: UAV Tracks Anything at Night with SAM-Powered Domain Adaptation](https://mlsanity.com/paper/2307.01024/) (2023)
- [A Survey on Trustworthiness in Foundation Models for Medical Image Analysis](https://mlsanity.com/paper/2407.15851/) (2024)
- [Interactive 3D Medical Image Segmentation with SAM 2](https://mlsanity.com/paper/2408.02635/) (2024)
- [mathrmSAMMed: A medical image annotation framework based on large vision model](https://mlsanity.com/paper/2307.05617/) (2023)
- [Open-NeRF: Towards Open Vocabulary NeRF Decomposition](https://mlsanity.com/paper/2310.16383/) (2023)
- [False Negative/Positive Control for SAM on Noisy Medical Images](https://mlsanity.com/paper/2308.10382/) (2023)
- [CC-SAM: SAM with Cross-feature Attention and Context for Ultrasound Image Segmentation](https://mlsanity.com/paper/2408.00181/) (2024)

## Abstract (arXiv)

Foundation models have taken over natural language processing and image generation domains due to the flexibility of prompting. With the recent introduction of the Segment Anything Model (SAM), this prompt-driven paradigm has entered image segmentation with a hitherto unexplored abundance of capabilities. The purpose of this paper is to conduct an initial evaluation of the out-of-the-box zero-shot capabilities of SAM for medical image segmentation, by evaluating its performance on an abdominal CT organ segmentation task, via point or bounding box based prompting. We show that SAM generalizes well to CT data, making it a potential catalyst for the advancement of semi-automatic segmentation tools for clinicians. We believe that this foundation model, while not reaching state-of-the-art segmentation performance in our investigations, can serve as a highly potent starting point for further adaptations of such models to the intricacies of the medical domain. Keywords: medical image segmentation, SAM, foundation models, zero-shot learning