# SAM4EM: Efficient memory-based two stage prompt-free segment anything model adapter for complex 3D neuroscience electron microscopy stacks

arXiv: https://arxiv.org/abs/2504.21544 · HTML: https://mlsanity.com/paper/2504.21544/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 39 references that are in the map.
- First posted on arXiv 30 Apr 2025.

## Builds on (most cited first, 17 of 17)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [Segment Anything in High Quality](https://mlsanity.com/paper/2306.01567/) (2023)
- [Customized Segment Anything Model for Medical Image Segmentation](https://mlsanity.com/paper/2304.13785/) (2023)
- [EfficientSAM: Leveraged Masked Image Pretraining for Efficient Segment Anything](https://mlsanity.com/paper/2312.00863/) (2023)
- [Transformer-Based Visual Segmentation: A Survey](https://mlsanity.com/paper/2304.09854/) (2023)
- [Segment Anything Model for Medical Image Segmentation: Current Applications and Future Directions](https://mlsanity.com/paper/2401.03495/) (2024)
- [Parameter-efficient Model Adaptation for Vision Transformers](https://mlsanity.com/paper/2203.16329/) (2022)
- [Unleashing the Potential of SAM for Medical Adaptation via Hierarchical Decoding](https://mlsanity.com/paper/2403.18271/) (2024)
- [UN-SAM: Universal Prompt-Free Segmentation for Generalized Nuclei Images](https://mlsanity.com/paper/2402.16663/) (2024)
- [FusionNet: A deep fully residual convolutional neural network for image segmentation in connectomics](https://mlsanity.com/paper/1612.05360/) (2016)
- [Parameter Efficient Fine-tuning via Cross Block Orchestration for Segment Anything Model](https://mlsanity.com/paper/2311.17112/) (2023)
- [ASAM: Boosting Segment Anything Model with Adversarial Tuning](https://mlsanity.com/paper/2405.00256/) (2024)
- [Efficient Low-rank Backpropagation for Vision Transformer Adaptation](https://mlsanity.com/paper/2309.15275/) (2023)
- [Segmentation in large-scale cellular electron microscopy with deep learning: A literature survey](https://mlsanity.com/paper/2206.07171/) (2022)
- [HIVE-Net: Centerline-Aware HIerarchical View-Ensemble Convolutional Network for Mitochondria Segmentation in EM Images](https://mlsanity.com/paper/2101.02877/) (2021)

## Built on by (most cited first, 2 of 2)

- [Brain3D: EEG-to-3D Decoding of Visual Representations via Multimodal Reasoning](https://mlsanity.com/paper/2604.08068/) (2026)
- [A Fully Automatic Pipeline for 3D Dendrite Instance Segmentation in SBF-SEM](https://mlsanity.com/paper/2610.03332/) (2026)

## Abstract (arXiv)

We present SAM4EM, a novel approach for 3D segmentation of complex neural structures in electron microscopy (EM) data by leveraging the Segment Anything Model (SAM) alongside advanced fine-tuning strategies. Our contributions include the development of a prompt-free adapter for SAM using two stage mask decoding to automatically generate prompt embeddings, a dual-stage fine-tuning method based on Low-Rank Adaptation (LoRA) for enhancing segmentation with limited annotated data, and a 3D memory attention mechanism to ensure segmentation consistency across 3D stacks. We further release a unique benchmark dataset for the segmentation of astrocytic processes and synapses. We evaluated our method on challenging neuroscience segmentation benchmarks, specifically targeting mitochondria, glia, and synapses, with significant accuracy improvements over state-of-the-art (SOTA) methods, including recent SAM-based adapters developed for the medical domain and other vision transformer-based approaches. Experimental results indicate that our approach outperforms existing solutions in the segmentation of complex processes like glia and post-synaptic densities. Our code and models are available at https://github.com/Uzshah/SAM4EM.