# Zero-shot capability of SAM-family models for bone segmentation in CT scans

arXiv: https://arxiv.org/abs/2411.08629 · HTML: https://mlsanity.com/paper/2411.08629/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 26 references that are in the map.
- First posted on arXiv 13 Nov 2024.

## Builds on (most cited first, 14 of 14)

- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [TotalSegmentator: robust segmentation of 104 anatomical structures in CT images](https://mlsanity.com/paper/2208.05868/) (2022)
- [Segment Anything Model for Medical Image Analysis: an Experimental Study](https://mlsanity.com/paper/2304.10517/) (2023)
- [VerSe: A Vertebrae Labelling and Segmentation Benchmark for Multi-detector CT Images](https://mlsanity.com/paper/2001.09193/) (2020)
- [Segment Anything Model for Medical Image Segmentation: Current Applications and Future Directions](https://mlsanity.com/paper/2401.03495/) (2024)
- [SA-Med2D-20M Dataset: Segment Anything in 2D Medical Imaging with 20 Million masks](https://mlsanity.com/paper/2311.11969/) (2023)
- [Deep Learning to Segment Pelvic Bones: Large-scale CT Datasets and Baseline Models](https://mlsanity.com/paper/2012.08721/) (2020)
- [Computer-Vision Benchmark Segment-Anything Model (SAM) in Medical Images: Accuracy in 12 Datasets](https://mlsanity.com/paper/2304.09324/) (2023)
- [Is SAM 2 Better than SAM in Medical Image Segmentation?](https://mlsanity.com/paper/2408.04212/) (2024)
- [Zero-shot performance of the Segment Anything Model (SAM) in 2D medical imaging: A comprehensive evaluation and practical guidelines](https://mlsanity.com/paper/2305.00109/) (2023)
- [Unleashing the Potential of SAM2 for Biomedical Images and Videos: A Survey](https://mlsanity.com/paper/2408.12889/) (2024)
- [Performance and Non-adversarial Robustness of the Segment Anything Model 2 in Surgical Video Segmentation](https://mlsanity.com/paper/2408.04098/) (2024)
- [Cross-Modal Vertical Federated Learning for MRI Reconstruction](https://mlsanity.com/paper/2306.02673/) (2023)
- [Seg-metrics: a Python package to compute segmentation metrics](https://mlsanity.com/paper/2403.07884/) (2024)

## Built on by (most cited first, 3 of 3)

- [Optimization of MedSAM model based on bounding box adaptive perturbation algorithm](https://mlsanity.com/paper/2503.19700/) (2025)
- [Prompting with the human-touch: evaluating model-sensitivity of foundation models for musculoskeletal CT segmentation](https://mlsanity.com/paper/2603.10541/) (2026)
- [SegBanana: Steering Unified Multimodal Models into Medical Segmenters](https://mlsanity.com/paper/2609.34235/) (2026)

## Abstract (arXiv)

The Segment Anything Model (SAM) and similar models build a family of promptable foundation models (FMs) for image and video segmentation. The object of interest is identified using prompts, such as bounding boxes or points. With these FMs becoming part of medical image segmentation, extensive evaluation studies are required to assess their strengths and weaknesses in clinical setting. Since the performance is highly dependent on the chosen prompting strategy, it is important to investigate different prompting techniques to define optimal guidelines that ensure effective use in medical image segmentation. Currently, no dedicated evaluation studies exist specifically for bone segmentation in CT scans, leaving a gap in understanding the performance for this task. Thus, we use non-iterative, ``optimal'' prompting strategies composed of bounding box, points and combinations to test the zero-shot capability of SAM-family models for bone CT segmentation on three different skeletal regions. Our results show that the best settings depend on the model type and size, dataset characteristics and objective to optimize. Overall, SAM and SAM2 prompted with a bounding box in combination with the center point for all the components of an object yield the best results across all tested settings. As the results depend on multiple factors, we provide a guideline for informed decision-making in 2D prompting with non-interactive, ''optimal'' prompts.