# Unleashing the Potential of SAM2 for Biomedical Images and Videos: A Survey

arXiv: https://arxiv.org/abs/2408.12889 · HTML: https://mlsanity.com/paper/2408.12889/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 31 references that are in the map.
- First posted on arXiv 23 Aug 2024.

## Builds on (most cited first, 24 of 24)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [A Survey on Deep Learning in Medical Image Analysis](https://mlsanity.com/paper/1702.05747/) (2017)
- [KAN: Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2404.19756/) (2024)
- [The Medical Segmentation Decathlon](https://mlsanity.com/paper/2106.05735/) (2021)
- [Customized Segment Anything Model for Medical Image Segmentation](https://mlsanity.com/paper/2304.13785/) (2023)
- [AbdomenCT-1K: Is Abdominal Organ Segmentation A Solved Problem?](https://mlsanity.com/paper/2010.14808/) (2020)
- [Medical SAM 2: Segment medical images as video via Segment Anything Model 2](https://mlsanity.com/paper/2408.00874/) (2024)
- [Large-scale Multi-Modal Pre-trained Models: A Comprehensive Survey](https://mlsanity.com/paper/2302.10035/) (2023)
- [Segment Anything Model for Medical Image Segmentation: Current Applications and Future Directions](https://mlsanity.com/paper/2401.03495/) (2024)
- [Surgical SAM 2: Real-time Segment Anything in Surgical Video by Efficient Frame Pruning](https://mlsanity.com/paper/2408.07931/) (2024)
- [Segment Anything in Medical Images and Videos: Benchmark and Deployment](https://mlsanity.com/paper/2408.03322/) (2024)
- [Bridging 2D and 3D Segmentation Networks for Computation Efficient Volumetric Medical Image Segmentation: An Empirical Study of 2.5D Solutions](https://mlsanity.com/paper/2010.06163/) (2020)
- [Segment anything model 2: an application to 2D and 3D medical images](https://mlsanity.com/paper/2408.00756/) (2024)
- [SAM 2 in Robotic Surgery: An Empirical Evaluation for Robustness and Generalization in Surgical Video Segmentation](https://mlsanity.com/paper/2408.04593/) (2024)
- [Interactive 3D Medical Image Segmentation with SAM 2](https://mlsanity.com/paper/2408.02635/) (2024)
- [Is SAM 2 Better than SAM in Medical Image Segmentation?](https://mlsanity.com/paper/2408.04212/) (2024)
- [Biomedical SAM 2: Segment Anything in Biomedical Images and Videos](https://mlsanity.com/paper/2408.03286/) (2024)
- [Performance and Non-adversarial Robustness of the Segment Anything Model 2 in Surgical Video Segmentation](https://mlsanity.com/paper/2408.04098/) (2024)
- [Zero-Shot Surgical Tool Segmentation in Monocular Video Using Segment Anything Model 2](https://mlsanity.com/paper/2408.01648/) (2024)
- [Novel adaptation of video segmentation to 3D MRI: efficient zero-shot knee segmentation with SAM2](https://mlsanity.com/paper/2408.04762/) (2024)
- [Polyp SAM 2: Advancing Zero shot Polyp Segmentation in Colorectal Cancer Detection](https://mlsanity.com/paper/2408.05892/) (2024)
- [Path-SAM2: Transfer SAM2 for digital pathology semantic segmentation](https://mlsanity.com/paper/2408.03651/) (2024)

## Built on by (most cited first, 14 of 14)

- [MedSAM2: Segment Anything in 3D Medical Images and Videos](https://mlsanity.com/paper/2504.03600/) (2025)
- [On Efficient Variants of Segment Anything Model: A Survey](https://mlsanity.com/paper/2410.04960/) (2024)
- [SAM2 for Image and Video Segmentation: A Comprehensive Survey](https://mlsanity.com/paper/2503.12781/) (2025)
- [Self-Prompting Polyp Segmentation in Colonoscopy using Hybrid Yolo-SAM 2 Model](https://mlsanity.com/paper/2409.09484/) (2024)
- [RadioActive: 3D Radiological Interactive Segmentation Benchmark](https://mlsanity.com/paper/2411.07885/) (2024)
- [Zero-shot capability of SAM-family models for bone segmentation in CT scans](https://mlsanity.com/paper/2411.08629/) (2024)
- [SAM2LoRA: Composite Loss-Guided, Parameter-Efficient Finetuning of SAM2 for Retinal Fundus Segmentation](https://mlsanity.com/paper/2510.10288/) (2025)
- [Bootstrapping Video Semantic Segmentation Model via Distillation-assisted Test-Time Adaptation](https://mlsanity.com/paper/2604.10950/) (2026)
- [SurgiSAM2: Fine-tuning a foundational model for surgical video anatomy segmentation and detection](https://mlsanity.com/paper/2503.03942/) (2025)
- [Advancements in Medical Image Classification through Fine-Tuning Natural Domain Foundation Models](https://mlsanity.com/paper/2505.19779/) (2025)
- [Med-PerSAM: One-Shot Visual Prompt Tuning for Personalized Segment Anything Model in Medical Domain](https://mlsanity.com/paper/2411.16123/) (2024)
- [SARTM: Segment Any RGB Thermal Model with Language aided Distillation](https://mlsanity.com/paper/2505.01950/) (2025)
- [Organoid Tracker: A SAM2-Powered Platform for Zero-shot Cyst Analysis in Human Kidney Organoid Videos](https://mlsanity.com/paper/2509.11063/) (2025)
- [Promptable Fire Segmentation: Unleashing SAM2's Potential for Real-Time Mobile Deployment with Strategic Bounding Box Guidance](https://mlsanity.com/paper/2510.21782/) (2025)

## Abstract (arXiv)

The unprecedented developments in segmentation foundational models have become a dominant force in the field of computer vision, introducing a multitude of previously unexplored capabilities in a wide range of natural images and videos. Specifically, the Segment Anything Model (SAM) signifies a noteworthy expansion of the prompt-driven paradigm into the domain of image segmentation. The recent introduction of SAM2 effectively extends the original SAM to a streaming fashion and demonstrates strong performance in video segmentation. However, due to the substantial distinctions between natural and medical images, the effectiveness of these models on biomedical images and videos is still under exploration. This paper presents an overview of recent efforts in applying and adapting SAM2 to biomedical images and videos. The findings indicate that while SAM2 shows promise in reducing annotation burdens and enabling zero-shot segmentation, its performance varies across different datasets and tasks. Addressing the domain gap between natural and medical images through adaptation and fine-tuning is essential to fully unleash SAM2's potential in clinical applications. To support ongoing research endeavors, we maintain an active repository that contains up-to-date SAM & SAM2-related papers and projects at https://github.com/YichiZhang98/SAM4MIS.