# On-the-Fly Test-time Adaptation for Medical Image Segmentation

arXiv: https://arxiv.org/abs/2203.05574 · HTML: https://mlsanity.com/paper/2203.05574/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 33 references that are in the map.
- First posted on arXiv 10 Mar 2022.

## Builds on (most cited first, 22 of 22)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [UNet++: A Nested U-Net Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1807.10165/) (2018)
- [3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation](https://mlsanity.com/paper/1606.06650/) (2016)
- [Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge](https://mlsanity.com/paper/1811.02629/) (2018)
- [Domain Adaptation for Medical Image Analysis: A Survey](https://mlsanity.com/paper/2102.09508/) (2021)
- [Medical Transformer: Gated Axial-Attention for Medical Image Segmentation](https://mlsanity.com/paper/2102.10662/) (2021)
- [Revisiting Batch Normalization For Practical Domain Adaptation](https://mlsanity.com/paper/1603.04779/) (2016)
- [Causality-inspired Single-source Domain Generalization for Medical Image Segmentation](https://mlsanity.com/paper/2111.12525/) (2021)
- [Test-Time Adaptable Neural Networks for Robust Medical Image Segmentation](https://mlsanity.com/paper/2004.04668/) (2020)
- [Unsupervised domain adaptation for medical imaging segmentation with self-ensembling](https://mlsanity.com/paper/1811.06042/) (2018)
- [Source-Free Domain Adaptive Fundus Image Segmentation with Denoised Pseudo-Labeling](https://mlsanity.com/paper/2109.09735/) (2021)
- [Source-Relaxed Domain Adaptation for Image Segmentation](https://mlsanity.com/paper/2005.03697/) (2020)
- [Unsupervised Domain Adaptation via Disentangled Representations: Application to Cross-Modality Liver Segmentation](https://mlsanity.com/paper/1907.13590/) (2019)
- [KiU-Net: Towards Accurate Segmentation of Biomedical Images using Over-complete Representations](https://mlsanity.com/paper/2006.04878/) (2020)
- [KiU-Net: Overcomplete Convolutional Architectures for Biomedical Image and Volumetric Segmentation](https://mlsanity.com/paper/2010.01663/) (2020)
- [Improving Robustness of Deep Learning Based Knee MRI Segmentation: Mixup and Adversarial Domain Adaptation](https://mlsanity.com/paper/1908.04126/) (2019)
- [Self domain adapted network](https://mlsanity.com/paper/2007.03162/) (2020)
- [Generalize Ultrasound Image Segmentation via Instant and Plug & Play Style Transfer](https://mlsanity.com/paper/2101.03711/) (2021)

## Built on by (most cited first, 30 of 34)

- [Towards Online Domain Adaptive Object Detection](https://mlsanity.com/paper/2204.05289/) (2022)
- [Protected Test-Time Adaptation via Online Entropy Matching: A Betting Approach](https://mlsanity.com/paper/2408.07511/) (2024)
- [DomainAdaptor: A Novel Approach to Test-time Adaptation](https://mlsanity.com/paper/2308.10297/) (2023)
- [From Denoising Training to Test-Time Adaptation: Enhancing Domain Generalization for Medical Image Segmentation](https://mlsanity.com/paper/2310.20271/) (2023)
- [Effective Restoration of Source Knowledge in Continual Test Time Adaptation](https://mlsanity.com/paper/2311.04991/) (2023)
- [PASS:Test-Time Prompting to Adapt Styles and Semantic Shapes in Medical Image Segmentation](https://mlsanity.com/paper/2410.01573/) (2024)
- [Medical Image Segmentation with InTEnt: Integrated Entropy Weighting for Single Image Test-Time Adaptation](https://mlsanity.com/paper/2402.09604/) (2024)
- [Do Vision Foundation Models Enhance Domain Generalization in Medical Image Segmentation?](https://mlsanity.com/paper/2409.07960/) (2024)
- [Keeping Medical AI Healthy and Trustworthy: A Review of Detection and Correction Methods for System Degradation](https://mlsanity.com/paper/2506.17442/) (2025)
- [Progressive Test Time Energy Adaptation for Medical Image Segmentation](https://mlsanity.com/paper/2503.16616/) (2025)
- [CLIPTTA: Robust Contrastive Vision-Language Test-Time Adaptation](https://mlsanity.com/paper/2507.14312/) (2025)
- [Open-Set Automatic Target Recognition](https://mlsanity.com/paper/2211.05883/) (2022)
- [Test-Time Adaptation for Nighttime Color-Thermal Semantic Segmentation](https://mlsanity.com/paper/2307.04470/) (2023)
- [Zero-shot Domain Generalization of Foundational Models for 3D Medical Image Segmentation: An Experimental Study](https://mlsanity.com/paper/2503.22862/) (2025)
- [SAM-aware Test-time Adaptation for Universal Medical Image Segmentation](https://mlsanity.com/paper/2506.05221/) (2025)
- [Generalizing Supervised Deep Learning MRI Reconstruction to Multiple and Unseen Contrasts using Meta-Learning Hypernetworks](https://mlsanity.com/paper/2307.06771/) (2023)
- [HyperCoil-Recon: A Hypernetwork-based Adaptive Coil Configuration Task Switching Network for MRI Reconstruction](https://mlsanity.com/paper/2308.04821/) (2023)
- [Adaptive Batch Normalization Networks for Adversarial Robustness](https://mlsanity.com/paper/2405.11708/) (2024)
- [MODL: Multilearner Online Deep Learning](https://mlsanity.com/paper/2405.18281/) (2024)
- [Domain Adaptive Diabetic Retinopathy Grading with Model Absence and Flowing Data](https://mlsanity.com/paper/2412.01203/) (2024)
- [ProKAN: Progressive Stacking of Kolmogorov-Arnold Networks for Efficient Liver Segmentation](https://mlsanity.com/paper/2412.19713/) (2024)
- [TTTFusion: A Test-Time Training-Based Strategy for Multimodal Medical Image Fusion in Surgical Robots](https://mlsanity.com/paper/2504.20362/) (2025)
- [Benchmarking and Explaining Deep Learning Cortical Lesion MRI Segmentation in Multiple Sclerosis](https://mlsanity.com/paper/2507.12092/) (2025)
- [HyperTTA: Test-Time Adaptation for Hyperspectral Image Classification under Distribution Shifts](https://mlsanity.com/paper/2509.08436/) (2025)
- [Towards Reliable Test-Time Adaptation: Style Invariance as a Correctness Likelihood](https://mlsanity.com/paper/2512.07390/) (2025)
- [Continual Test-Time Adaptation in Computer Vision: Methods, Benchmarks, and Future Directions](https://mlsanity.com/paper/2607.08164/) (2026)
- [CONTRAST: Continual Multi-source Adaptation to Dynamic Distributions](https://mlsanity.com/paper/2401.02561/) (2024)
- [Plug-and-Play Transformer Modules for Test-Time Adaptation](https://mlsanity.com/paper/2401.04130/) (2024)
- [CrowdTransfer: Enabling Crowd Knowledge Transfer in AIoT Community](https://mlsanity.com/paper/2407.06485/) (2024)
- [Boundary-Aware Test-Time Adaptation for Zero-Shot Medical Image Segmentation](https://mlsanity.com/paper/2512.04520/) (2025)

## Abstract (arXiv)

One major problem in deep learning-based solutions for medical imaging is the drop in performance when a model is tested on a data distribution different from the one that it is trained on. Adapting the source model to target data distribution at test-time is an efficient solution for the data-shift problem. Previous methods solve this by adapting the model to target distribution by using techniques like entropy minimization or regularization. In these methods, the models are still updated by back-propagation using an unsupervised loss on complete test data distribution. In real-world clinical settings, it makes more sense to adapt a model to a new test image on-the-fly and avoid model update during inference due to privacy concerns and lack of computing resource at deployment. To this end, we propose a new setting - On-the-Fly Adaptation which is zero-shot and episodic (i.e., the model is adapted to a single image at a time and also does not perform any back-propagation during test-time). To achieve this, we propose a new framework called Adaptive UNet where each convolutional block is equipped with an adaptive batch normalization layer to adapt the features with respect to a domain code. The domain code is generated using a pre-trained encoder trained on a large corpus of medical images. During test-time, the model takes in just the new test image and generates a domain code to adapt the features of source model according to the test data. We validate the performance on both 2D and 3D data distribution shifts where we get a better performance compared to previous test-time adaptation methods. Code is available at https://github.com/jeya-maria-jose/On-The-Fly-Adaptation