# Test-time Adaptation for Foundation Medical Segmentation Model without Parametric Updates

arXiv: https://arxiv.org/abs/2504.02008 · HTML: https://mlsanity.com/paper/2504.02008/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 33 references that are in the map.
- First posted on arXiv 2 Apr 2025.

## Builds on (most cited first, 24 of 24)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentation](https://mlsanity.com/paper/2304.12620/) (2023)
- [Efficient Test-Time Model Adaptation without Forgetting](https://mlsanity.com/paper/2204.02610/) (2022)
- [MEMO: Test Time Robustness via Adaptation and Augmentation](https://mlsanity.com/paper/2110.09506/) (2021)
- [Domain Generalization for Medical Imaging Classification with Linear-Dependency Regularization](https://mlsanity.com/paper/2009.12829/) (2020)
- [DoFE: Domain-oriented Feature Embedding for Generalizable Fundus Image Segmentation on Unseen Datasets](https://mlsanity.com/paper/2010.06208/) (2020)
- [Input Augmentation with SAM: Boosting Medical Image Segmentation with Segmentation Foundation Model](https://mlsanity.com/paper/2304.11332/) (2023)
- [Content Adaptive Optimization for Neural Image Compression](https://mlsanity.com/paper/1906.01223/) (2019)
- [Test-Time Training Can Close the Natural Distribution Shift Performance Gap in Deep Learning Based Compressed Sensing](https://mlsanity.com/paper/2204.07204/) (2022)
- [From Denoising Training to Test-Time Adaptation: Enhancing Domain Generalization for Medical Image Segmentation](https://mlsanity.com/paper/2310.20271/) (2023)
- [Efficient Test-Time Adaptation for Super-Resolution with Second-Order Degradation and Reconstruction](https://mlsanity.com/paper/2310.19011/) (2023)
- [FocusNetv2: Imbalanced Large and Small Organ Segmentation with Adversarial Shape Constraint for Head and Neck CT Images](https://mlsanity.com/paper/2104.01771/) (2021)
- [PASS:Test-Time Prompting to Adapt Styles and Semantic Shapes in Medical Image Segmentation](https://mlsanity.com/paper/2410.01573/) (2024)
- [Universal Deep Image Compression via Content-Adaptive Optimization with Adapters](https://mlsanity.com/paper/2211.00918/) (2022)
- [Test Time Adaptation for Blind Image Quality Assessment](https://mlsanity.com/paper/2307.14735/) (2023)
- [Gradient Alignment Improves Test-Time Adaptation for Medical Image Segmentation](https://mlsanity.com/paper/2408.07343/) (2024)
- [Dynamic Low-Rank Instance Adaptation for Universal Neural Image Compression](https://mlsanity.com/paper/2308.07733/) (2023)
- [Medical Image Segmentation with InTEnt: Integrated Entropy Weighting for Single Image Test-Time Adaptation](https://mlsanity.com/paper/2402.09604/) (2024)
- [Adapting the Segment Anything Model During Usage in Novel Situations](https://mlsanity.com/paper/2404.08421/) (2024)
- [Mitigating Training Imbalance in LLM Fine-Tuning via Selective Parameter Merging](https://mlsanity.com/paper/2410.03743/) (2024)
- [Test-time adaptation for image compression with distribution regularization](https://mlsanity.com/paper/2410.12191/) (2024)
- [Improving Segment Anything on the Fly: Auxiliary Online Learning and Adaptive Fusion for Medical Image Segmentation](https://mlsanity.com/paper/2406.00956/) (2024)
- [A SAM-guided and Match-based Semi-Supervised Segmentation Framework for Medical Imaging](https://mlsanity.com/paper/2411.16949/) (2024)
- [Semi-Supervised Medical Image Segmentation via Knowledge Mining from Large Models](https://mlsanity.com/paper/2503.06816/) (2025)

## Built on by (most cited first, 5 of 5)

- [SAM-aware Test-time Adaptation for Universal Medical Image Segmentation](https://mlsanity.com/paper/2506.05221/) (2025)
- [Efficient Test-Time Adaptation through Latent Subspace Coefficients Search](https://mlsanity.com/paper/2510.11068/) (2025)
- [SPACE: SPike-Aware Consistency Enhancement for Test-Time Adaptation in Spiking Neural Networks](https://mlsanity.com/paper/2504.02298/) (2025)
- [Boundary-Aware Test-Time Adaptation for Zero-Shot Medical Image Segmentation](https://mlsanity.com/paper/2512.04520/) (2025)
- [EviATTA: Evidential Active Test-Time Adaptation for Medical Segment Anything Models](https://mlsanity.com/paper/2603.14666/) (2026)

## Abstract (arXiv)

Foundation medical segmentation models, with MedSAM being the most popular, have achieved promising performance across organs and lesions. However, MedSAM still suffers from compromised performance on specific lesions with intricate structures and appearance, as well as bounding box prompt-induced perturbations. Although current test-time adaptation (TTA) methods for medical image segmentation may tackle this issue, partial (e.g., batch normalization) or whole parametric updates restrict their effectiveness due to limited update signals or catastrophic forgetting in large models. Meanwhile, these approaches ignore the computational complexity during adaptation, which is particularly significant for modern foundation models. To this end, our theoretical analyses reveal that directly refining image embeddings is feasible to approach the same goal as parametric updates under the MedSAM architecture, which enables us to realize high computational efficiency and segmentation performance without the risk of catastrophic forgetting. Under this framework, we propose to encourage maximizing factorized conditional probabilities of the posterior prediction probability using a proposed distribution-approximated latent conditional random field loss combined with an entropy minimization loss. Experiments show that we achieve about 3% Dice score improvements across three datasets while reducing computational complexity by over 7 times.