# From Adaptation to Generalization: Adaptive Visual Prompting for Medical Image Segmentation

arXiv: https://arxiv.org/abs/2604.17455 · HTML: https://mlsanity.com/paper/2604.17455/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 48 references that are in the map.
- First posted on arXiv 19 Apr 2026.

## Builds on (most cited first, 27 of 27)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [Visual Prompt Tuning](https://mlsanity.com/paper/2203.12119/) (2022)
- [Kvasir-SEG: A Segmented Polyp Dataset](https://mlsanity.com/paper/1911.07069/) (2019)
- [PraNet: Parallel Reverse Attention Network for Polyp Segmentation](https://mlsanity.com/paper/2006.11392/) (2020)
- [Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net](https://mlsanity.com/paper/1807.09441/) (2018)
- [Road Extraction by Deep Residual U-Net](https://mlsanity.com/paper/1711.10684/) (2017)
- [REFUGE Challenge: A Unified Framework for Evaluating Automated Methods for Glaucoma Assessment from Fundus Photographs](https://mlsanity.com/paper/1910.03667/) (2019)
- [Medical Image Segmentation Review: The success of U-Net](https://mlsanity.com/paper/2211.14830/) (2022)
- [Transfer Learning for Domain Adaptation in MRI: Application in Brain Lesion Segmentation](https://mlsanity.com/paper/1702.07841/) (2017)
- [Video Prediction Recalling Long-term Motion Context via Memory Alignment Learning](https://mlsanity.com/paper/2104.00924/) (2021)
- [RDumb: A simple approach that questions our progress in continual test-time adaptation](https://mlsanity.com/paper/2306.05401/) (2023)
- [NeoUNet: Towards accurate colon polyp segmentation and neoplasm detection](https://mlsanity.com/paper/2107.05023/) (2021)
- [One-Prompt to Segment All Medical Images](https://mlsanity.com/paper/2305.10300/) (2023)
- [Each Test Image Deserves A Specific Prompt: Continual Test-Time Adaptation for 2D Medical Image Segmentation](https://mlsanity.com/paper/2311.18363/) (2023)
- [Modality-agnostic Domain Generalizable Medical Image Segmentation by Multi-Frequency in Multi-Scale Attention](https://mlsanity.com/paper/2405.06284/) (2024)
- [MAPSeg: Unified Unsupervised Domain Adaptation for Heterogeneous Medical Image Segmentation Based on 3D Masked Autoencoding and Pseudo-Labeling](https://mlsanity.com/paper/2303.09373/) (2023)
- [FVP: Fourier Visual Prompting for Source-Free Unsupervised Domain Adaptation of Medical Image Segmentation](https://mlsanity.com/paper/2304.13672/) (2023)
- [Source-Free Domain Adaptive Fundus Image Segmentation with Class-Balanced Mean Teacher](https://mlsanity.com/paper/2307.09973/) (2023)
- [Prompt Tuning of Deep Neural Networks for Speaker-adaptive Visual Speech Recognition](https://mlsanity.com/paper/2302.08102/) (2023)
- [Source-Free Domain Adaptation for Medical Image Segmentation via Prototype-Anchored Feature Alignment and Contrastive Learning](https://mlsanity.com/paper/2307.09769/) (2023)
- [Complementary Domain Adaptation and Generalization for Unsupervised Continual Domain Shift Learning](https://mlsanity.com/paper/2303.15833/) (2023)
- [Context-Aware Pseudo-Label Refinement for Source-Free Domain Adaptive Fundus Image Segmentation](https://mlsanity.com/paper/2308.07731/) (2023)
- [Medical Image Segmentation via Single-Source Domain Generalization with Random Amplitude Spectrum Synthesis](https://mlsanity.com/paper/2409.04768/) (2024)
- [A2XP: Towards Private Domain Generalization](https://mlsanity.com/paper/2311.10339/) (2023)
- [FrogDogNet: Fourier frequency Retained visual prompt Output Guidance for Domain Generalization of CLIP in Remote Sensing](https://mlsanity.com/paper/2504.16433/) (2025)

## Abstract (arXiv)

Visual prompting has emerged as a powerful method for adapting pre-trained models to new domains without updating model parameters. However, existing prompting methods typically optimize a single prompt per domain and apply it uniformly to all inputs, limiting their ability to generalize under intra and inter-domain variability, which is especially critical in the medical field. To address this, we propose APEX, an Adaptive Prompt EXtraction framework that retrieves input-specific prompts from a learnable prompt memory. The memory stores diverse, domain-discriminative prompt representations and is queried via domain features extracted from the Fourier spectrum. To learn robust and discriminative domain features, we introduce a novel Low-Frequency Feature Contrastive (LFC) learning framework that clusters representations from the same domain while separating those from different domains. Extensive experiments on two medical segmentation tasks demonstrate that APEX significantly improves generalization across both seen and unseen domains. Furthermore, it complements any existing backbones and consistently enhances performance, confirming its effectiveness as a plug-and-play prompting solution in medical fields. The code is available at https://github.com/cetinkayaevren/apex/