mlsanityData to 14 Oct 2026

Segment Anything in Medical Images and Videos: Benchmark and Deployment

arXiv 2408.03322 ↗First posted 6 Aug 2024eess.IV cs.CVcited by 42

Not on the map. Why?

Abstract (arXiv)

Recent advances in segmentation foundation models have enabled accurate and efficient segmentation across a wide range of natural images and videos, but their utility to medical data remains unclear. In this work, we first present a comprehensive benchmarking of the Segment Anything Model 2 (SAM2) across 11 medical image modalities and videos and point out its strengths and weaknesses by comparing it to SAM1 and MedSAM. Then, we develop a transfer learning pipeline and demonstrate SAM2 can be quickly adapted to medical domain by fine-tuning. Furthermore, we implement SAM2 as a 3D slicer plugin and Gradio API for efficient 3D image and video segmentation. The code has been made publicly available at urlhttps://github.com/bowang-lab/MedSAM.

Key facts. Cited by 42 papers in the map (data through 14 Oct 2026). Builds on 33 of its 55 references that are in the map. First posted on arXiv 6 Aug 2024.

Builds on

33 of 55 references.

References · 33

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2023Segment Anythingcited by 8,234
  4. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
  5. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  6. 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
  7. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 929
  8. 2023Segment Everything Everywhere All at Oncecited by 548
  9. 2018Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challengecited by 511
  10. 2019The Liver Tumor Segmentation Benchmark (LiTS)cited by 509
  11. 2023Segment Anything in High Qualitycited by 376
  12. 2023Segment Anything Model for Medical Image Analysis: an Experimental Studycited by 291
21 more
  1. 2023Hiera: A Hierarchical Vision Transformer without the Bells-and-Whistlescited by 290
  2. 2019Embracing Imperfect Datasets: A Review of Deep Learning Solutions for Medical Image Segmentationcited by 260
  3. 2019Deep Learning for Segmentation using an Open Large-Scale Dataset in 2D Echocardiographycited by 244
  4. 2020AbdomenCT-1K: Is Abdominal Organ Segmentation A Solved Problem?cited by 224
  5. 2019The state of the art in kidney and kidney tumor segmentation in contrast-enhanced CT imaging: Results of the KiTS19 Challengecited by 190
  6. 2019Gradio: Hassle-Free Sharing and Testing of ML Models in the Wildcited by 178
  7. 2024Medical SAM 2: Segment medical images as video via Segment Anything Model 2cited by 154
  8. 2023On the Challenges and Perspectives of Foundation Models for Medical Image Analysiscited by 135
  9. 20202018 Robotic Scene Segmentation Challengecited by 124
  10. 2024BiomedParse: a biomedical foundation model for image parsing of everything everywhere all at oncecited by 111
  11. 2023SegVol: Universal and Interactive Volumetric Medical Image Segmentationcited by 87
  12. 2022Video Polyp Segmentation: A Deep Learning Perspectivecited by 82
  13. 2023MA-SAM: Modality-agnostic SAM Adaptation for 3D Medical Image Segmentationcited by 82
  14. 20233DSAM-adapter: Holistic adaptation of SAM from 2D to 3D for promptable tumor segmentationcited by 70
  15. 2024EVF-SAM: Early Vision-Language Fusion for Text-Prompted Segment Anything Modelcited by 65
  16. 2023Large-Vocabulary Segmentation for Medical Images with Text Promptscited by 62
  17. 2020Towards Data-Efficient Learning: A Benchmark for COVID-19 CT Lung and Infection Segmentationcited by 53
  18. 2023The Multi-modality Cell Segmentation Challenge: Towards Universal Solutionscited by 39
  19. 2024Segment anything model 2: an application to 2D and 3D medical imagescited by 28
  20. 2024Interactive 3D Medical Image Segmentation with SAM 2cited by 17
  21. 2024SAM-Med3D-MoE: Towards a Non-Forgetting Segment Anything Model via Mixture of Experts for 3D Medical Image Segmentationcited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 13; 2025: 20; 2026: 9.

42 papers.

Sort:

By area:

  1. 2025MedSAM2: Segment Anything in 3D Medical Images and Videoscited by 96
  2. 2024SAM2-Adapter: Evaluating & Adapting Segment Anything 2 in Downstream Tasks: Camouflage, Shadow, Medical Image Segmentation, and Morecited by 60
  3. 2024Unleashing the Potential of SAM2 for Biomedical Images and Videos: A Surveycited by 14
  4. 2025SAM2 for Image and Video Segmentation: A Comprehensive Surveycited by 13
  5. 2024When SAM2 Meets Video Camouflaged Object Segmentation: A Comprehensive Evaluation and Adaptationcited by 11
  6. 2025SegAnyPET: Universal Promptable Segmentation from Positron Emission Tomography Imagescited by 11
  7. 2025Adapting Vision Foundation Models for Real-time Ultrasound Image Segmentationcited by 9
  8. 2024Polyp SAM 2: Advancing Zero shot Polyp Segmentation in Colorectal Cancer Detectioncited by 8
  9. 2025Surgical Scene Understanding in the Era of Foundation AI Models: A Comprehensive Reviewcited by 8
  10. 2024RevSAM2: Prompt SAM2 for Medical Image Segmentation via Reverse-Propagation without Fine-tuningcited by 7
  11. 2025SAM3-Adapter: Efficient Adaptation of Segment Anything 3 for Camouflage Object Segmentation, Shadow Detection, and Medical Image Segmentationcited by 7
  12. 2025MedicoSAM: Robust Improvement of SAM for Medical Imagingcited by 6
30 more
  1. 2024A Short Review and Evaluation of SAM2's Performance in 3D CT Image Segmentationcited by 5
  2. 2024Evaluation Study on SAM 2 for Class-agnostic Instance-level Segmentationcited by 5
  3. 2025Can Foundation Models Really Segment Tumors? A Benchmarking Odyssey in Lung CT Imagingcited by 5
  4. 2026MedVL-SAM2: A unified 3D medical vision-language model for multimodal reasoning and prompt-driven segmentationcited by 5
  5. 2024Path-SAM2: Transfer SAM2 for digital pathology semantic segmentationcited by 4
  6. 2024DRL-STNet: Unsupervised Domain Adaptation for Cross-modality Medical Image Segmentation via Disentangled Representation Learningcited by 3
  7. 2025Agglomerating Large Vision Encoders via Distillation for VFSS Segmentationcited by 3
  8. 2025Auto-nnU-Net: Towards Automated Medical Image Segmentationcited by 3
  9. 2024SAM-Swin: SAM-Driven Dual-Swin Transformers with Adaptive Lesion Enhancement for Laryngo-Pharyngeal Tumor Detectioncited by 2
  10. 2025Promptable cancer segmentation using minimal expert-curated datacited by 2
  11. 2025Comparing SAM 2 and SAM 3 for Zero-Shot Segmentation of 3D Medical Datacited by 2
  12. 2024Adaptive Interactive Segmentation for Multimodal Medical Imaging via Selection Enginecited by 1
  13. 2024First-frame Supervised Video Polyp Segmentation via Propagative and Semantic Dual-teacher Networkcited by 1
  14. 2025SAM2-SGP: Enhancing SAM2 for Medical Image Segmentation via Support-Set Guided Promptingcited by 1
  15. 2025Segment Anything for Cell Trackingcited by 1
  16. 2026SurgNavAR: An Augmented Reality Surgical Navigation Framework for Optical See-Through Head Mounted Displayscited by 1
  17. 2024Zero-shot 3D Segmentation of Abdominal Organs in CT Scans Using Segment Anything Model 2: Adapting Video Tracking Capabilities for 3D Medical Imagingcited by 0
  18. 2025AI-Driven MRI-based Brain Tumour Segmentation Benchmarkingcited by 0
  19. 2025FreeVPS: Repurposing Training-Free SAM2 for Generalizable Video Polyp Segmentationcited by 0
  20. 2025Organoid Tracker: A SAM2-Powered Platform for Zero-shot Cyst Analysis in Human Kidney Organoid Videoscited by 0
  21. 2025EMeRALDS: Electronic Medical Record Driven Automated Lung Nodule Detection and Classification in Thoracic CT Imagescited by 0
  22. 2025Towards Better Ultrasound Video Segmentation Foundation Model: An Empirical study on SAM2 Finetuning from Data Perspectivecited by 0
  23. 2025Boundary-Aware Test-Time Adaptation for Zero-Shot Medical Image Segmentationcited by 0
  24. 2026Sub-Region-Aware Modality Fusion and Adaptive Prompting for Multi-Modal Brain Tumor Segmentationcited by 0
  25. 2026A Hybrid Mamba-SAM Architecture for Efficient 3D Medical Image Segmentationcited by 0
  26. 2026Enhancing MedSAM with a Lightweight Box Predictor for Medical Image Segmentationcited by 0
  27. 2026SAMRI-3D: Adapting SAM2 for 3D MRI Segmentation with Global Volume Tokenscited by 0
  28. 2026Report Supervisioncited by 0
  29. 2026LeCor: Learning to Be Corrected by Meta-Learned Test-Time Training for Interactive 3D Lung-Tumour Segmentationcited by 0
  30. 2026IMVS: Interactive Medical Volume Segmentation with Test-Time Adaptation - A New Method for Annotating Radiology Datasetscited by 0