mlsanityData to 7 Oct 2026

Segment Anything Model for Medical Image Segmentation: Current Applications and Future Directions

arXiv 2401.03495 ↗First posted 7 Jan 2024eess.IV cs.CVcited by 97

Not on the map. Why?

Abstract (arXiv)

Due to the inherent flexibility of prompting, foundation models have emerged as the predominant force in the fields of natural language processing and computer vision. The recent introduction of the Segment Anything Model (SAM) signifies a noteworthy expansion of the prompt-driven paradigm into the domain of image segmentation, thereby introducing a plethora of previously unexplored capabilities. However, the viability of its application to medical image segmentation remains uncertain, given the substantial distinctions between natural and medical images. In this work, we provide a comprehensive overview of recent endeavors aimed at extending the efficacy of SAM to medical image segmentation tasks, encompassing both empirical benchmarking and methodological adaptations. Additionally, we explore potential avenues for future research directions in SAM's role within medical image segmentation. While direct application of SAM to medical image segmentation does not yield satisfactory performance on multi-modal and multi-target medical datasets so far, numerous insights gleaned from these efforts serve as valuable guidance for shaping the trajectory of foundational models in the realm of medical image analysis. To support ongoing research endeavors, we maintain an active repository that contains an up-to-date paper list and a succinct summary of open-source projects at https://github.com/YichiZhang98/SAM4MIS.

Key facts. Cited by 97 papers in the map (data through 7 Oct 2026). Builds on 75 of its 104 references that are in the map. First posted on arXiv 7 Jan 2024.

Builds on

75 of 104 references.

References · 75

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  6. 2023GPT-4 Technical Reportcited by 18,132
  7. 2023Segment Anythingcited by 8,158
  8. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  9. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  10. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  11. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,284
  12. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
38 more
  1. 2017A Survey on Deep Learning in Medical Image Analysiscited by 1,182
  2. 2021The Medical Segmentation Decathloncited by 620
  3. 2022TotalSegmentator: robust segmentation of 104 anatomical structures in CT imagescited by 597
  4. 2019Kvasir-SEG: A Segmented Polyp Datasetcited by 588
  5. 2023Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentationcited by 485
  6. 2022AMOS: A Large-Scale Abdominal Multi-Organ Benchmark for Versatile Medical Image Segmentationcited by 347
  7. 2023Segment Anything Model for Medical Image Analysis: an Experimental Studycited by 291
  8. 2023Customized Segment Anything Model for Medical Image Segmentationcited by 277
  9. 2019Embracing Imperfect Datasets: A Review of Deep Learning Solutions for Medical Image Segmentationcited by 258
  10. 2020A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promisescited by 230
  11. 2020AbdomenCT-1K: Is Abdominal Organ Segmentation A Solved Problem?cited by 222
  12. 2016A Benchmark for Endoluminal Scene Segmentation of Colonoscopy Imagescited by 220
  13. 2023Segment Anything Model (SAM) for Digital Pathology: Assess Zero-shot Segmentation on Whole Slide Imagingcited by 161
  14. 2023Large-scale Multi-Modal Pre-trained Models: A Comprehensive Surveycited by 153
  15. 2023On the Challenges and Perspectives of Foundation Models for Medical Image Analysiscited by 135
  16. 20202018 Robotic Scene Segmentation Challengecited by 123
  17. 2023SAM-Med2Dcited by 123
  18. 2023Segment Anything Is Not Always Perfect: An Investigation of SAM on Different Real-world Applicationscited by 121
  19. 2023Foundational Models Defining a New Era in Vision: A Survey and Outlookcited by 109
  20. 2023SAM-Med3D: Towards General-purpose Segmentation Models for Volumetric Medical Imagescited by 101
  21. 2021Multi-modal Understanding and Generation for Medical Images and Text via Vision-Language Pre-Trainingcited by 96
  22. 2023SegVol: Universal and Interactive Volumetric Medical Image Segmentationcited by 86
  23. 2023SAM on Medical Images: A Comprehensive Study on Three Prompt Modescited by 81
  24. 2023SAM Struggles in Concealed Scenes -- Empirical Study on Segment Anythingcited by 75
  25. 2022Learning with Limited Annotations: A Survey on Deep Semi-Supervised Learning for Medical Image Segmentationcited by 74
  26. 2023AutoSAM: Adapting SAM to Medical Images by Overloading the Prompt Encodercited by 70
  27. 20233DSAM-adapter: Holistic adaptation of SAM from 2D to 3D for promptable tumor segmentationcited by 70
  28. 2023Polyp-SAM: Transfer SAM for Polyp Segmentationcited by 64
  29. 2023Large-Vocabulary Segmentation for Medical Images with Text Promptscited by 60
  30. 2023Can SAM Segment Polyps?cited by 59
  31. 2023Input Augmentation with SAM: Boosting Medical Image Segmentation with Segmentation Foundation Modelcited by 56
  32. 2023SA-Med2D-20M Dataset: Segment Anything in 2D Medical Imaging with 20 Million maskscited by 54
  33. 2023When SAM Meets Medical Images: An Investigation of Segment Anything Model (SAM) on Multi-phase Liver Tumor Segmentationcited by 52
  34. 2023Beyond Adapting SAM: Towards End-to-End Ultrasound Image Segmentation via Auto Promptingcited by 40
  35. 2023SAM3D: Segment Anything Model in Volumetric Medical Imagescited by 34
  36. 2023SkinSAM: Empowering Skin Cancer Segmentation with Segment Anything Modelcited by 32
  37. 2023Artificial General Intelligence for Medical Imaging Analysiscited by 32
  38. 2023All-in-SAM: from Weak Annotation to Pixel-wise Nuclei Segmentation with Prompt-based Finetuningcited by 32

50 most cited of 75 All 75 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 36; 2025: 48; 2026: 13.

97 papers.

Sort:

By area:

  1. 2024A Comprehensive Survey of Foundation Models in Medicinecited by 56
  2. 2024Image Segmentation in Foundation Model Era: A Surveycited by 35
  3. 2024How to build the best medical image segmentation algorithm using foundation models: a comprehensive empirical study with Segment Anything Modelcited by 32
  4. 2024Interactive Medical Image Segmentation: A Benchmark Dataset and Baselinecited by 30
  5. 2024UN-SAM: Universal Prompt-Free Segmentation for Generalized Nuclei Imagescited by 22
  6. 2024Serp-Mamba: Advancing High-Resolution Retinal Vessel Segmentation with Selective State-Space Modelcited by 20
  7. 2024On Efficient Variants of Segment Anything Model: A Surveycited by 16
  8. 2025Dino U-Net: Exploiting High-Fidelity Dense Features from Foundation Models for Medical Image Segmentationcited by 16
  9. 2024FairDiff: Fair Segmentation with Point-Image Diffusioncited by 15
  10. 2024FastSAM3D: An Efficient Segment Anything Model for 3D Volumetric Medical Imagescited by 14
  11. 2024Unleashing the Potential of SAM2 for Biomedical Images and Videos: A Surveycited by 14
  12. 2024ProtoSAM: One-Shot Medical Image Segmentation With Foundational Modelscited by 12
38 more
  1. 2024ASI-Seg: Audio-Driven Surgical Instrument Segmentation with Surgeon Intention Understandingcited by 12
  2. 2025SegAnyPET: Universal Promptable Segmentation from Positron Emission Tomography Imagescited by 11
  3. 2025Diffusion-empowered AutoPrompt MedSAMcited by 10
  4. 2025Vision-Language Model for Object Detection and Segmentation: A Review and Evaluationcited by 10
  5. 2024SAM-MPA: Applying SAM to Few-shot Medical Image Segmentation using Mask Propagation and Auto-promptingcited by 9
  6. 2025No time to train! Training-Free Reference-Based Instance Segmentationcited by 9
  7. 2025Surgical Scene Understanding in the Era of Foundation AI Models: A Comprehensive Reviewcited by 8
  8. 2025Recent Advances in Medical Imaging Segmentation: A Surveycited by 8
  9. 2025MedicoSAM: Robust Improvement of SAM for Medical Imagingcited by 6
  10. 2025Benchmarking MedMNIST dataset on real quantum hardwarecited by 6
  11. 2025OCRT: Boosting Foundation Models in the Open World with Object-Concept-Relation Triadcited by 6
  12. 2025RadSAM: Segmenting 3D radiological images with a 2D promptable modelcited by 6
  13. 2025From Classical Machine Learning to Emerging Foundation Models: Review on Multimodal Data Integration for Cancer Researchcited by 6
  14. 2024NuSegDG: Integration of Heterogeneous Space and Gaussian Kernel for Domain-Generalized Nuclei Segmentationcited by 5
  15. 2024MSDNet: Multi-Scale Decoder for Few-Shot Semantic Segmentation via Transformer-Guided Prototypingcited by 5
  16. 2024Zero-Shot Automatic Annotation and Instance Segmentation using LLM-Generated Datasets: Eliminating Field Imaging and Manual Annotation for Deep Learning Model Developmentcited by 5
  17. 2025Few-Shot Adaptation of Training-Free Foundation Model for 3D Medical Image Segmentationcited by 5
  18. 2024MaskSAM: Towards Auto-prompt SAM with Mask Classification for Volumetric Medical Image Segmentationcited by 4
  19. 2024Beyond Pixel-Wise Supervision for Medical Image Segmentation: From Traditional Models to Foundation Modelscited by 4
  20. 2024Self-supervised Mamba-based Mastoidectomy Shape Prediction for Cochlear Implant Surgerycited by 4
  21. 2025MedM-VL: What Makes a Good Medical LVLM?cited by 4
  22. 2024ShapeMamba-EM: Fine-Tuning Foundation Model with Local Shape Descriptors and Mamba Blocks for 3D EM Image Segmentationcited by 3
  23. 2024TuneVLSeg: Prompt Tuning Benchmark for Vision-Language Segmentation Modelscited by 3
  24. 2024RClicks: Realistic Click Simulation for Benchmarking Interactive Segmentationcited by 3
  25. 2024Zero-shot capability of SAM-family models for bone segmentation in CT scanscited by 3
  26. 2024Exploring Aleatoric Uncertainty in Object Detection via Vision Foundation Modelscited by 3
  27. 2025Federated Client-tailored Adapter for Medical Image Segmentationcited by 3
  28. 2025SAM-aware Test-time Adaptation for Universal Medical Image Segmentationcited by 3
  29. 2025Flow Stochastic Segmentation Networkscited by 3
  30. 2025Impact of Clinical Image Quality on Efficient Foundation Model Finetuningcited by 3
  31. 2025Adaptation of Foundation Models for Medical Image Analysis: Strategies, Challenges, and Future Directionscited by 3
  32. 2025On the Utility of Foundation Models for Fast MRI: Vision-Language-Guided Image Reconstructioncited by 3
  33. 2025Bridging the Perception-Cognition Gap:Re-engineering SAM2 with Hilbert-Mamba for Robust VLM-based Medical Diagnosiscited by 3
  34. 2024MedSAGa: Few-shot Memory Efficient Medical Image Segmentation using Gradient Low-Rank Projection in SAMcited by 2
  35. 2024Zero-Shot Pupil Segmentation with SAM 2: A Case Study of Over 14 Million Imagescited by 2
  36. 2024Assessing Foundational Medical 'Segment Anything' (Med-SAM1, Med-SAM2) Deep Learning Models for Left Atrial Segmentation in 3D LGE MRIcited by 2
  37. 2025SALT: Parameter-Efficient Fine-Tuning via Singular Value Adaptation with Low-Rank Transformationcited by 2
  38. 2025Generalist Models in Medical Image Segmentation: A Survey and Performance Comparison with Task-Specific Approachescited by 2

50 most cited of 97 All 97 (JSON)