# Adaptive Prompt Learning with SAM for Few-shot Scanning Probe Microscope Image Segmentation arXiv: https://arxiv.org/abs/2410.12562 ยท HTML: https://mlsanity.com/paper/2410.12562/ ## Key facts - Cited by 1 papers in the map (data through 14 Oct 2026). - Builds on 30 of its 56 references that are in the map. - First posted on arXiv 16 Oct 2024. ## Builds on (most cited first, 30 of 30) - [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020) - [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020) - [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015) - [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023) - [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023) - [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021) - [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023) - [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014) - [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019) - [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021) - [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016) - [Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentation](https://mlsanity.com/paper/2304.12620/) (2023) - [Segment Anything in High Quality](https://mlsanity.com/paper/2306.01567/) (2023) - [One-Shot Learning for Semantic Segmentation](https://mlsanity.com/paper/1709.03410/) (2017) - [How to train your MAML](https://mlsanity.com/paper/1810.09502/) (2018) - [Prior Guided Feature Enrichment Network for Few-Shot Segmentation](https://mlsanity.com/paper/2008.01449/) (2020) - [Adaptive Prototype Learning and Allocation for Few-Shot Segmentation](https://mlsanity.com/paper/2104.01893/) (2021) - [Semantic-SAM: Segment and Recognize Anything at Any Granularity](https://mlsanity.com/paper/2307.04767/) (2023) - [SG-One: Similarity Guidance Network for One-Shot Semantic Segmentation](https://mlsanity.com/paper/1810.09091/) (2018) - [Self-Supervision with Superpixels: Training Few-shot Medical Image Segmentation without Annotation](https://mlsanity.com/paper/2007.09886/) (2020) - [SAM-CLIP: Merging Vision Foundation Models towards Semantic and Spatial Understanding](https://mlsanity.com/paper/2310.15308/) (2023) - [Recurrent Mask Refinement for Few-Shot Medical Image Segmentation](https://mlsanity.com/paper/2108.00622/) (2021) - [SkinSAM: Empowering Skin Cancer Segmentation with Segment Anything Model](https://mlsanity.com/paper/2304.13973/) (2023) - [Self-Sampling Meta SAM: Enhancing Few-shot Medical Image Segmentation with Meta-Learning](https://mlsanity.com/paper/2308.16466/) (2023) - [Few-Shot Microscopy Image Cell Segmentation](https://mlsanity.com/paper/2007.01671/) (2020) - [Improving the Segmentation of Scanning Probe Microscope Images using Convolutional Neural Networks](https://mlsanity.com/paper/2008.12371/) (2020) - [Segmentation of Scanning Tunneling Microscopy Images Using Variational Methods and Empirical Wavelets](https://mlsanity.com/paper/1804.08890/) (2018) - [SAMDA: Leveraging SAM on Few-Shot Domain Adaptation for Electronic Microscopy Segmentation](https://mlsanity.com/paper/2403.07951/) (2024) ## Built on by (most cited first, 1 of 1) - [Prompt Engineering in Segment Anything Model: Methodologies, Applications, and Emerging Challenges](https://mlsanity.com/paper/2507.09562/) (2025) ## Abstract (arXiv) The Segment Anything Model (SAM) has demonstrated strong performance in image segmentation of natural scene images. However, its effectiveness diminishes markedly when applied to specific scientific domains, such as Scanning Probe Microscope (SPM) images. This decline in accuracy can be attributed to the distinct data distribution and limited availability of the data inherent in the scientific images. On the other hand, the acquisition of adequate SPM datasets is both time-intensive and laborious as well as skill-dependent. To address these challenges, we propose an Adaptive Prompt Learning with SAM (APL-SAM) framework tailored for few-shot SPM image segmentation. Our approach incorporates two key innovations to enhance SAM: 1) An Adaptive Prompt Learning module leverages few-shot embeddings derived from limited support set to learn adaptively central representatives, serving as visual prompts. This innovation eliminates the need for time-consuming online user interactions for providing prompts, such as exhaustively marking points and bounding boxes slice by slice; 2) A multi-source, multi-level mask decoder specifically designed for few-shot SPM image segmentation is introduced, which can effectively capture the correspondence between the support and query images. To facilitate comprehensive training and evaluation, we introduce a new dataset, SPM-Seg, curated for SPM image segmentation. Extensive experiments on this dataset reveal that the proposed APL-SAM framework significantly outperforms the original SAM, achieving over a 30% improvement in terms of Dice Similarity Coefficient with only one-shot guidance. Moreover, APL-SAM surpasses state-of-the-art few-shot segmentation methods and even fully supervised approaches in performance. Code and dataset used in this study will be made available upon acceptance.