# Segment Anything Model (SAM) for Radiation Oncology

arXiv: https://arxiv.org/abs/2306.11730 · HTML: https://mlsanity.com/paper/2306.11730/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 48 of its 116 references that are in the map.
- First posted on arXiv 20 Jun 2023.

## Builds on (most cited first, 30 of 48)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [A Survey on Deep Learning in Medical Image Analysis](https://mlsanity.com/paper/1702.05747/) (2017)
- [PaLM 2 Technical Report](https://mlsanity.com/paper/2305.10403/) (2023)
- [Resolution-robust Large Mask Inpainting with Fourier Convolutions](https://mlsanity.com/paper/2109.07161/) (2021)
- [Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentation](https://mlsanity.com/paper/2304.12620/) (2023)
- [Segment Anything Model for Medical Image Analysis: an Experimental Study](https://mlsanity.com/paper/2304.10517/) (2023)
- [Inpaint Anything: Segment Anything Meets Image Inpainting](https://mlsanity.com/paper/2304.06790/) (2023)
- [U-Net and its variants for medical image segmentation: theory and applications](https://mlsanity.com/paper/2011.01118/) (2020)
- [Segment Anything Model (SAM) for Digital Pathology: Assess Zero-shot Segmentation on Whole Slide Imaging](https://mlsanity.com/paper/2304.04155/) (2023)
- [Bayesian Convolutional Neural Networks with Bernoulli Approximate Variational Inference](https://mlsanity.com/paper/1506.02158/) (2015)
- [Summary of ChatGPT-Related Research and Perspective Towards the Future of Large Language Models](https://mlsanity.com/paper/2304.01852/) (2023)
- [DeID-GPT: Zero-shot Medical Text De-Identification by GPT-4](https://mlsanity.com/paper/2303.11032/) (2023)
- [Segment Anything Is Not Always Perfect: An Investigation of SAM on Different Real-world Applications](https://mlsanity.com/paper/2304.05750/) (2023)
- [Self-supervised learning methods and applications in medical imaging analysis: A survey](https://mlsanity.com/paper/2109.08685/) (2021)
- [SAM Struggles in Concealed Scenes -- Empirical Study on Segment Anything](https://mlsanity.com/paper/2304.06022/) (2023)
- [Polyp-SAM: Transfer SAM for Polyp Segmentation](https://mlsanity.com/paper/2305.00293/) (2023)
- [Can SAM Segment Polyps?](https://mlsanity.com/paper/2304.07583/) (2023)
- [An Iterative Optimizing Framework for Radiology Report Summarization with ChatGPT](https://mlsanity.com/paper/2304.08448/) (2023)
- [AnatomyNet: Deep Learning for Fast and Fully Automated Whole-volume Segmentation of Head and Neck Anatomy](https://mlsanity.com/paper/1808.05238/) (2018)
- [When Brain-inspired AI Meets AGI](https://mlsanity.com/paper/2303.15935/) (2023)
- [When SAM Meets Medical Images: An Investigation of Segment Anything Model (SAM) on Multi-phase Liver Tumor Segmentation](https://mlsanity.com/paper/2304.08506/) (2023)
- [Evaluating Large Language Models on a Highly-specialized Topic, Radiation Oncology Physics](https://mlsanity.com/paper/2304.01938/) (2023)
- [Radiology-GPT: A Large Language Model for Radiology](https://mlsanity.com/paper/2306.08666/) (2023)
- [Deep reinforcement learning in medical imaging: A literature review](https://mlsanity.com/paper/2103.05115/) (2021)

## Built on by (most cited first, 24 of 24)

- [Segment Anything Model for Medical Image Segmentation: Current Applications and Future Directions](https://mlsanity.com/paper/2401.03495/) (2024)
- [Large Language Models for Robotics: Opportunities, Challenges, and Perspectives](https://mlsanity.com/paper/2401.04334/) (2024)
- [MA-SAM: Modality-agnostic SAM Adaptation for 3D Medical Image Segmentation](https://mlsanity.com/paper/2309.08842/) (2023)
- [Understanding LLMs: A Comprehensive Overview from Training to Inference](https://mlsanity.com/paper/2401.02038/) (2024)
- [Unleashing the Potential of SAM for Medical Adaptation via Hierarchical Decoding](https://mlsanity.com/paper/2403.18271/) (2024)
- [Radiology-Llama2: Best-in-Class Large Language Model for Radiology](https://mlsanity.com/paper/2309.06419/) (2023)
- [SAMAug: Point Prompt Augmentation for Segment Anything Model](https://mlsanity.com/paper/2307.01187/) (2023)
- [Deep Interactive Segmentation of Medical Images: A Systematic Review and Taxonomy](https://mlsanity.com/paper/2311.13964/) (2023)
- [Fantastic Animals and Where to Find Them: Segment Any Marine Animal with Dual SAM](https://mlsanity.com/paper/2404.04996/) (2024)
- [LLM-driven Multimodal Target Volume Contouring in Radiation Oncology](https://mlsanity.com/paper/2311.01908/) (2023)
- [Holistic Evaluation of GPT-4V for Biomedical Imaging](https://mlsanity.com/paper/2312.05256/) (2023)
- [RadOnc-GPT: A Large Language Model for Radiation Oncology](https://mlsanity.com/paper/2309.10160/) (2023)
- [MAS-SAM: Segment Any Marine Animal with Aggregated Features](https://mlsanity.com/paper/2404.15700/) (2024)
- [Evaluating Large Language Models for Radiology Natural Language Processing](https://mlsanity.com/paper/2307.13693/) (2023)
- [Vision Foundation Models in Medical Image Analysis: Advances and Challenges](https://mlsanity.com/paper/2502.14584/) (2025)
- [Unleashing the Potential of Vision-Language Pre-Training for 3D Zero-Shot Lesion Segmentation via Mask-Attribute Alignment](https://mlsanity.com/paper/2410.15744/) (2024)
- [Hierarchical Semantic Tree Concept Whitening for Interpretable Image Classification](https://mlsanity.com/paper/2307.04343/) (2023)
- [Benchmarking a foundation LLM on its ability to re-label structure names in accordance with the AAPM TG-263 report](https://mlsanity.com/paper/2310.03874/) (2023)
- [Gaze-Assisted Medical Image Segmentation](https://mlsanity.com/paper/2410.17920/) (2024)
- [PharmacyGPT: The AI Pharmacist](https://mlsanity.com/paper/2307.10432/) (2023)
- [Boosting Federated Domain Generalization: Understanding the Role of Advanced Pre-Trained Architectures](https://mlsanity.com/paper/2409.13527/) (2024)
- [Mixture of Multicenter Experts in Multimodal AI for Debiased Radiotherapy Target Delineation](https://mlsanity.com/paper/2410.00046/) (2024)
- [LithoSeg: A Coarse-to-Fine Framework for High-Precision Lithography Segmentation](https://mlsanity.com/paper/2511.12005/) (2025)
- [ScribbleDose: Scribble-Guided Dose Prediction in Radiotherapy](https://mlsanity.com/paper/2605.11555/) (2026)

## Abstract (arXiv)

In this study, we evaluate the performance of the Segment Anything Model (SAM) in clinical radiotherapy. Our results indicate that SAM's 'segment anything' mode can achieve clinically acceptable segmentation results in most organs-at-risk (OARs) with Dice scores higher than 0.7. SAM's 'box prompt' mode further improves the Dice scores by 0.1 to 0.5. Considering the size of the organ and the clarity of its boundary, SAM displays better performance for large organs with clear boundaries but performs worse for smaller organs with unclear boundaries. Given that SAM, a model pre-trained purely on natural images, can handle the delineation of OARs from medical images with clinically acceptable accuracy, these results highlight SAM's robust generalization capabilities with consistent accuracy in automatic segmentation for radiotherapy. In other words, SAM can achieve delineation of different OARs at different sites using a generic automatic segmentation model. SAM's generalization capabilities across different disease sites suggest that it is technically feasible to develop a generic model for automatic segmentation in radiotherapy.