# I-MedSAM: Implicit Medical Image Segmentation with Segment Anything

arXiv: https://arxiv.org/abs/2311.17081 · HTML: https://mlsanity.com/paper/2311.17081/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 56 references that are in the map.
- First posted on arXiv 28 Nov 2023.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [UNETR: Transformers for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.10504/) (2021)
- [Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentation](https://mlsanity.com/paper/2304.12620/) (2023)
- [Res2Net: A New Multi-scale Backbone Architecture](https://mlsanity.com/paper/1904.01169/) (2019)
- [PraNet: Parallel Reverse Attention Network for Polyp Segmentation](https://mlsanity.com/paper/2006.11392/) (2020)
- [AMOS: A Large-Scale Abdominal Multi-Organ Benchmark for Versatile Medical Image Segmentation](https://mlsanity.com/paper/2206.08023/) (2022)
- [Implicit Functions in Feature Space for 3D Shape Reconstruction and Completion](https://mlsanity.com/paper/2003.01456/) (2020)
- [Customized Segment Anything Model for Medical Image Segmentation](https://mlsanity.com/paper/2304.13785/) (2023)
- [Training behavior of deep neural network in frequency domain](https://mlsanity.com/paper/1807.01251/) (2018)
- [SAM-Med2D](https://mlsanity.com/paper/2308.16184/) (2023)
- [Implicit Neural Representation in Medical Imaging: A Comparative Survey](https://mlsanity.com/paper/2307.16142/) (2023)
- [Intriguing Findings of Frequency Selection for Image Deblurring](https://mlsanity.com/paper/2111.11745/) (2021)
- [A Comprehensive Study on Colorectal Polyp Segmentation with ResUNet++, Conditional Random Field and Test-Time Augmentation](https://mlsanity.com/paper/2107.12435/) (2021)
- [Zero-shot Medical Image Translation via Frequency-Guided Diffusion Models](https://mlsanity.com/paper/2304.02742/) (2023)
- [SAM3D: Segment Anything Model in Volumetric Medical Images](https://mlsanity.com/paper/2309.03493/) (2023)
- [Single-subject Multi-contrast MRI Super-resolution via Implicit Neural Representations](https://mlsanity.com/paper/2303.15065/) (2023)
- [Learning Implicit Feature Alignment Function for Semantic Segmentation](https://mlsanity.com/paper/2206.08655/) (2022)
- [NISF: Neural Implicit Segmentation Functions](https://mlsanity.com/paper/2309.08643/) (2023)
- [Implicit field learning for unsupervised anomaly detection in medical images](https://mlsanity.com/paper/2106.05214/) (2021)
- [NTO3D: Neural Target Object 3D Reconstruction with Segment Anything](https://mlsanity.com/paper/2309.12790/) (2023)
- [DiffDP: Radiotherapy Dose Prediction via a Diffusion Model](https://mlsanity.com/paper/2307.09794/) (2023)
- [Implicit Anatomical Rendering for Medical Image Segmentation with Stochastic Experts](https://mlsanity.com/paper/2304.03209/) (2023)
- [OSS-Net: Memory Efficient High Resolution Semantic Segmentation of 3D Medical Data](https://mlsanity.com/paper/2110.10640/) (2021)

## Built on by (most cited first, 22 of 22)

- [PRISM: A Promptable and Robust Interactive Segmentation Model with Visual Prompts](https://mlsanity.com/paper/2404.15028/) (2024)
- [MAS-SAM: Segment Any Marine Animal with Aggregated Features](https://mlsanity.com/paper/2404.15700/) (2024)
- [CRISP-SAM2: SAM2 with Cross-Modal Interaction and Semantic Prompting for Multi-Organ Segmentation](https://mlsanity.com/paper/2506.23121/) (2025)
- [MedSAM-Agent: Empowering Interactive Medical Image Segmentation with Multi-turn Agentic Reinforcement Learning](https://mlsanity.com/paper/2602.03320/) (2026)
- [Mask-Enhanced Segment Anything Model for Tumor Lesion Semantic Segmentation](https://mlsanity.com/paper/2403.05912/) (2024)
- [Privacy-Preserving SAM Quantization for Efficient Edge Intelligence in Healthcare](https://mlsanity.com/paper/2410.01813/) (2024)
- [Integrating Deep Learning in Cardiology: A Comprehensive Review of Atrial Fibrillation, Left Atrial Scar Segmentation, and the Frontiers of State-of-the-Art Techniques](https://mlsanity.com/paper/2407.09561/) (2024)
- [DNRSelect: Active Best View Selection for Deferred Neural Rendering](https://mlsanity.com/paper/2501.12150/) (2025)
- [Improvise, Adapt, Overcome -- Telescopic Adapters for Efficient Fine-tuning of Vision Language Models in Medical Imaging](https://mlsanity.com/paper/2512.13855/) (2025)
- [Adaptive Mix for Semi-Supervised Medical Image Segmentation](https://mlsanity.com/paper/2407.21586/) (2024)
- [Frequency Domain Enhanced U-Net for Low-Frequency Information-Rich Image Segmentation in Surgical and Deep-Sea Exploration Robots](https://mlsanity.com/paper/2502.03829/) (2025)
- [QFGN: A Quantum Approach to High-Fidelity Implicit Neural Representations](https://mlsanity.com/paper/2504.19053/) (2025)
- [MedCore: Boundary-Preserving Medical Core Pruning for MedSAM](https://mlsanity.com/paper/2605.13688/) (2026)
- [MedSAM-CA: A CNN-Augmented ViT with Attention-Enhanced Multi-Scale Fusion for Medical Image Segmentation](https://mlsanity.com/paper/2506.23700/) (2025)
- [SAMora: Enhancing SAM through Hierarchical Self-Supervised Pre-Training for Medical Images](https://mlsanity.com/paper/2511.08626/) (2025)
- [TGC-Net: A Structure-Aware and Semantically-Aligned Framework for Text-Guided Medical Image Segmentation](https://mlsanity.com/paper/2512.21135/) (2025)
- [HFP-SAM: Hierarchical Frequency Prompted SAM for Efficient Marine Animal Segmentation](https://mlsanity.com/paper/2603.12708/) (2026)
- [Segment Anything with Robust Uncertainty-Accuracy Correlation](https://mlsanity.com/paper/2605.10603/) (2026)
- [SparseStreet: Sparse Gaussian Splatting for Real-Time Street Scene Simulation](https://mlsanity.com/paper/2606.03909/) (2026)
- [PEFT-MedSAM: Efficient Fine-Tuning of Medical Foundation Models for Explainable Skin Lesion Segmentation](https://mlsanity.com/paper/2606.18707/) (2026)
- [Frequency and Edge-Guided Segment Anything Model for Remote Sensing Image Semantic Segmentation](https://mlsanity.com/paper/2608.15054/) (2026)
- [Feature Reconfiguration With Visual Prior for Medical Lesion Segmentation](https://mlsanity.com/paper/2609.03535/) (2026)

## Abstract (arXiv)

With the development of Deep Neural Networks (DNNs), many efforts have been made to handle medical image segmentation. Traditional methods such as nnUNet train specific segmentation models on the individual datasets. Plenty of recent methods have been proposed to adapt the foundational Segment Anything Model (SAM) to medical image segmentation. However, they still focus on discrete representations to generate pixel-wise predictions, which are spatially inflexible and scale poorly to higher resolution. In contrast, implicit methods learn continuous representations for segmentation, which is crucial for medical image segmentation. In this paper, we propose I-MedSAM, which leverages the benefits of both continuous representations and SAM, to obtain better cross-domain ability and accurate boundary delineation. Since medical image segmentation needs to predict detailed segmentation boundaries, we designed a novel adapter to enhance the SAM features with high-frequency information during Parameter-Efficient Fine-Tuning (PEFT). To convert the SAM features and coordinates into continuous segmentation output, we utilize Implicit Neural Representation (INR) to learn an implicit segmentation decoder. We also propose an uncertainty-guided sampling strategy for efficient learning of INR. Extensive evaluations on 2D medical image segmentation tasks have shown that our proposed method with only 1.6M trainable parameters outperforms existing methods including discrete and implicit methods. The code will be available at: https://github.com/ucwxb/I-MedSAM.