# Comprehensive Multimodal Segmentation in Medical Imaging: Combining YOLOv8 with SAM and HQ-SAM Models

arXiv: https://arxiv.org/abs/2310.12995 · HTML: https://mlsanity.com/paper/2310.12995/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 19 references that are in the map.
- First posted on arXiv 4 Oct 2023.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation](https://mlsanity.com/paper/1606.06650/) (2016)
- [Attention U-Net: Learning Where to Look for the Pancreas](https://mlsanity.com/paper/1804.03999/) (2018)
- [Segment Anything in High Quality](https://mlsanity.com/paper/2306.01567/) (2023)
- [Improving Semantic Segmentation via Decoupled Body and Edge Supervision](https://mlsanity.com/paper/2007.10035/) (2020)
- [Deep Learning for Multi-Task Medical Image Segmentation in Multiple Modalities](https://mlsanity.com/paper/1704.03379/) (2017)

## Built on by (most cited first, 18 of 18)

- [Segment Anything Model for Medical Image Segmentation: Current Applications and Future Directions](https://mlsanity.com/paper/2401.03495/) (2024)
- [Foundation Model for Advancing Healthcare: Challenges, Opportunities, and Future Directions](https://mlsanity.com/paper/2404.03264/) (2024)
- [Test-Time Adaptation with SaLIP: A Cascade of SAM and CLIP for Zero shot Medical Image Segmentation](https://mlsanity.com/paper/2404.06362/) (2024)
- [DC-SAM: In-Context Segment Anything in Images and Videos via Dual Consistency](https://mlsanity.com/paper/2504.12080/) (2025)
- [A Systematic Review of Deep Learning-based Research on Radiology Report Generation](https://mlsanity.com/paper/2311.14199/) (2023)
- [SAM-MPA: Applying SAM to Few-shot Medical Image Segmentation using Mask Propagation and Auto-prompting](https://mlsanity.com/paper/2411.17363/) (2024)
- [Recent Advances in Medical Imaging Segmentation: A Survey](https://mlsanity.com/paper/2505.09274/) (2025)
- [Auto-Generating Weak Labels for Real & Synthetic Data to Improve Label-Scarce Medical Image Segmentation](https://mlsanity.com/paper/2404.17033/) (2024)
- [Segment Anything for Video: A Comprehensive Review of Video Object Segmentation and Tracking from Past to Future](https://mlsanity.com/paper/2507.22792/) (2025)
- [FusionSAM: Visual Multi-Modal Learning with Segment Anything](https://mlsanity.com/paper/2408.13980/) (2024)
- [Semi-Supervised Medical Image Segmentation via Knowledge Mining from Large Models](https://mlsanity.com/paper/2503.06816/) (2025)
- [RobustMedSAM: Degradation-Resilient Medical Image Segmentation via Robust Foundation Model Adaptation](https://mlsanity.com/paper/2604.09814/) (2026)
- [Multi-Stage Segmentation and Cascade Classification Methods for Improving Cardiac MRI Analysis](https://mlsanity.com/paper/2412.09386/) (2024)
- [AutoMiSeg: Automatic Medical Image Segmentation via Test-Time Adaptation of Foundation Models](https://mlsanity.com/paper/2505.17931/) (2025)
- [Geometric Feature Prompting of Image Segmentation Models](https://mlsanity.com/paper/2505.21644/) (2025)
- [Real-Time Threaded Houbara Detection and Segmentation for Wildlife Conservation using Mobile Platforms](https://mlsanity.com/paper/2510.03501/) (2025)
- [Neural expressiveness for beyond importance model compression](https://mlsanity.com/paper/2512.06440/) (2025)
- [Dissecting Model Failures in Abdominal Aortic Aneurysm Segmentation through Explainability-Driven Analysis](https://mlsanity.com/paper/2603.24801/) (2026)

## Abstract (arXiv)

This paper introduces a comprehensive approach for segmenting regions of interest (ROI) in diverse medical imaging datasets, encompassing ultrasound, CT scans, and X-ray images. The proposed method harnesses the capabilities of the YOLOv8 model for approximate boundary box detection across modalities, alongside the Segment Anything Model (SAM) and High Quality (HQ) SAM for fully automatic and precise segmentation. To generate boundary boxes, the YOLOv8 model was trained using a limited set of 100 images and masks from each modality. The results obtained from our approach are extensively computed and analyzed, demonstrating its effectiveness and potential in medical image analysis. Various evaluation metrics, including precision, recall, F1 score, and Dice Score, were employed to quantify the accuracy of the segmentation results. A comparative analysis was conducted to assess the individual and combined performance of the YOLOv8, YOLOv8+SAM, and YOLOv8+HQ-SAM models. The results indicate that the SAM model performs better than the other two models, exhibiting higher segmentation accuracy and overall performance. While HQ-SAM offers potential advantages, its incremental gains over the standard SAM model may not justify the additional computational cost. The YOLOv8+SAM model shows promise for enhancing medical image segmentation and its clinical implications.