# Evaluation and improvement of Segment Anything Model for interactive histopathology image segmentation

arXiv: https://arxiv.org/abs/2310.10493 · HTML: https://mlsanity.com/paper/2310.10493/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 26 references that are in the map.
- First posted on arXiv 16 Oct 2023.

## Builds on (most cited first, 14 of 14)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Reviving Iterative Training with Mask Guidance for Interactive Segmentation](https://mlsanity.com/paper/2102.06583/) (2021)
- [Deep Interactive Object Selection](https://mlsanity.com/paper/1603.04042/) (2016)
- [f-BRS: Rethinking Backpropagating Refinement for Interactive Segmentation](https://mlsanity.com/paper/2001.10331/) (2020)
- [SimpleClick: Interactive Image Segmentation with Simple Vision Transformers](https://mlsanity.com/paper/2210.11006/) (2022)
- [FocalClick: Towards Practical Interactive Image Segmentation](https://mlsanity.com/paper/2204.02574/) (2022)
- [AutoSAM: Adapting SAM to Medical Images by Overloading the Prompt Encoder](https://mlsanity.com/paper/2306.06370/) (2023)
- [Iteratively Trained Interactive Segmentation](https://mlsanity.com/paper/1805.04398/) (2018)
- [PseudoClick: Interactive Image Segmentation with Click Imitation](https://mlsanity.com/paper/2207.05282/) (2022)
- [DeSAM: Decoupled Segment Anything Model for Generalizable Medical Image Segmentation](https://mlsanity.com/paper/2306.00499/) (2023)
- [CGAM: Click-Guided Attention Module for Interactive Pathology Image Segmentation via Backpropagating Refinement](https://mlsanity.com/paper/2307.01015/) (2023)

## Built on by (most cited first, 1 of 1)

- [CellPilot: A unified approach to automatic and interactive segmentation in histopathology](https://mlsanity.com/paper/2411.15514/) (2024)

## Abstract (arXiv)

With the emergence of the Segment Anything Model (SAM) as a foundational model for image segmentation, its application has been extensively studied across various domains, including the medical field. However, its potential in the context of histopathology data, specifically in region segmentation, has received relatively limited attention. In this paper, we evaluate SAM's performance in zero-shot and fine-tuned scenarios on histopathology data, with a focus on interactive segmentation. Additionally, we compare SAM with other state-of-the-art interactive models to assess its practical potential and evaluate its generalization capability with domain adaptability. In the experimental results, SAM exhibits a weakness in segmentation performance compared to other models while demonstrating relative strengths in terms of inference time and generalization capability. To improve SAM's limited local refinement ability and to enhance prompt stability while preserving its core strengths, we propose a modification of SAM's decoder. The experimental results suggest that the proposed modification is effective to make SAM useful for interactive histology image segmentation. The code is available at urlhttps://github.com/hvcl/SAMInteractiveHistopathology