# Adapting Segment Anything Model (SAM) through Prompt-based Learning for Enhanced Protein Identification in Cryo-EM Micrographs

arXiv: https://arxiv.org/abs/2311.16140 · HTML: https://mlsanity.com/paper/2311.16140/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 16 references that are in the map.
- First posted on arXiv 4 Nov 2023.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Florence: A New Foundation Model for Computer Vision](https://mlsanity.com/paper/2111.11432/) (2021)
- [Segment Anything in Medical Images](https://mlsanity.com/paper/2304.12306/) (2023)
- [Semi-supervised Sequence Learning](https://mlsanity.com/paper/1511.01432/) (2015)
- [Segment Anything Is Not Always Perfect: An Investigation of SAM on Different Real-world Applications](https://mlsanity.com/paper/2304.05750/) (2023)
- [Generative Visual Prompt: Unifying Distributional Control of Pre-Trained Generative Models](https://mlsanity.com/paper/2209.06970/) (2022)
- [Positive-unlabeled convolutional neural networks for particle picking in cryo-electron micrographs](https://mlsanity.com/paper/1803.08207/) (2018)
- [DeepPicker: a Deep Learning Approach for Fully Automated Particle Picking in Cryo-EM](https://mlsanity.com/paper/1605.01838/) (2016)
- [A deep convolutional neural network approach to single-particle recognition in cryo-electron microscopy](https://mlsanity.com/paper/1605.05543/) (2016)
- [APPLE Picker: Automatic Particle Picking, a Low-Effort Cryo-EM Framework](https://mlsanity.com/paper/1802.00469/) (2018)

## Built on by (most cited first, 3 of 3)

- [Advanced POD-Based Performance Evaluation of Classifiers Applied to Human Driver Lane Changing Prediction](https://mlsanity.com/paper/2408.15722/) (2024)
- [Promoting SAM for Camouflaged Object Detection via Selective Key Point-based Guidance](https://mlsanity.com/paper/2505.09123/) (2025)
- [TopoFuse: Topology-Aware Tri-Planar Fusion for 3D Cryo-Electron Tomography Segmentation](https://mlsanity.com/paper/2609.29717/) (2026)

## Abstract (arXiv)

Cryo-electron microscopy (cryo-EM) remains pivotal in structural biology, yet the task of protein particle picking, integral for 3D protein structure construction, is laden with manual inefficiencies. While recent AI tools such as Topaz and crYOLO are advancing the field, they do not fully address the challenges of cryo-EM images, including low contrast, complex shapes, and heterogeneous conformations. This study explored prompt-based learning to adapt the state-of-the-art image segmentation foundation model Segment Anything Model (SAM) for cryo-EM. This focus was driven by the desire to optimize model performance with a small number of labeled data without altering pre-trained parameters, aiming for a balance between adaptability and foundational knowledge retention. Through trials with three prompt-based learning strategies, namely head prompt, prefix prompt, and encoder prompt, we observed enhanced performance and reduced computational requirements compared to the fine-tuning approach. This work not only highlights the potential of prompting SAM in protein identification from cryo-EM micrographs but also suggests its broader promise in biomedical image segmentation and object detection.