# A Field of Experts Prior for Adapting Neural Networks at Test Time

arXiv: https://arxiv.org/abs/2202.05271 · HTML: https://mlsanity.com/paper/2202.05271/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 65 references that are in the map.
- First posted on arXiv 10 Feb 2022.

## Builds on (most cited first, 30 of 30)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [Deep Anomaly Detection with Outlier Exposure](https://mlsanity.com/paper/1812.04606/) (2018)
- [Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2002.08546/) (2020)
- [Do Deep Generative Models Know What They Don't Know?](https://mlsanity.com/paper/1810.09136/) (2018)
- [Improving robustness against common corruptions by covariate shift adaptation](https://mlsanity.com/paper/2006.16971/) (2020)
- [Domain Generalization via Model-Agnostic Learning of Semantic Features](https://mlsanity.com/paper/1910.13580/) (2019)
- [Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning?](https://mlsanity.com/paper/1706.00712/) (2017)
- [Causality matters in medical imaging](https://mlsanity.com/paper/1912.08142/) (2019)
- [Unsupervised domain adaptation in brain lesion segmentation with adversarial networks](https://mlsanity.com/paper/1612.08894/) (2016)
- [Shape-aware Meta-learning for Generalizing Prostate MRI Segmentation to Unseen Domains](https://mlsanity.com/paper/2007.02035/) (2020)
- [Unsupervised Learning for Fast Probabilistic Diffeomorphic Registration](https://mlsanity.com/paper/1805.04605/) (2018)
- [Standardized Assessment of Automatic Segmentation of White Matter Hyperintensities and Results of the WMH Segmentation Challenge](https://mlsanity.com/paper/1904.00682/) (2019)
- [Test-Time Adaptable Neural Networks for Robust Medical Image Segmentation](https://mlsanity.com/paper/2004.04668/) (2020)
- [Density of States Estimation for Out-of-Distribution Detection](https://mlsanity.com/paper/2006.09273/) (2020)
- [Time-Dependent Deep Image Prior for Dynamic MRI](https://mlsanity.com/paper/1910.01684/) (2019)
- [SynSeg-Net: Synthetic Segmentation Without Target Modality Ground Truth](https://mlsanity.com/paper/1810.06498/) (2018)
- [Source-Relaxed Domain Adaptation for Image Segmentation](https://mlsanity.com/paper/2005.03697/) (2020)
- [Source-Free Adaptation to Measurement Shift via Bottom-Up Feature Restoration](https://mlsanity.com/paper/2107.05446/) (2021)
- [A Lifelong Learning Approach to Brain MR Segmentation Across Scanners and Protocols](https://mlsanity.com/paper/1805.10170/) (2018)
- [Training Deep Learning Based Denoisers without Ground Truth Data](https://mlsanity.com/paper/1803.01314/) (2018)
- [Image-Adaptive GAN based Reconstruction](https://mlsanity.com/paper/1906.05284/) (2019)
- [Semi-supervised Meta-learning with Disentanglement for Domain-generalised Medical Image Segmentation](https://mlsanity.com/paper/2106.13292/) (2021)
- [Correction Filter for Single Image Super-Resolution: Robustifying Off-the-Shelf Deep Super-Resolvers](https://mlsanity.com/paper/1912.00157/) (2019)
- [Source-free Domain Adaptation via Distributional Alignment by Matching Batch Normalization Statistics](https://mlsanity.com/paper/2101.10842/) (2021)
- [Model Adaptation for Inverse Problems in Imaging](https://mlsanity.com/paper/2012.00139/) (2020)
- [Limitations of Post-Hoc Feature Alignment for Robustness](https://mlsanity.com/paper/2103.05898/) (2021)

## Built on by (most cited first, 1 of 1)

- [A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts](https://mlsanity.com/paper/2303.15361/) (2023)

## Abstract (arXiv)

Performance of convolutional neural networks (CNNs) in image analysis tasks is often marred in the presence of acquisition-related distribution shifts between training and test images. Recently, it has been proposed to tackle this problem by fine-tuning trained CNNs for each test image. Such test-time-adaptation (TTA) is a promising and practical strategy for improving robustness to distribution shifts as it requires neither data sharing between institutions nor annotating additional data. Previous TTA methods use a helper model to increase similarity between outputs and/or features extracted from a test image with those of the training images. Such helpers, which are typically modeled using CNNs, can be task-specific and themselves vulnerable to distribution shifts in their inputs. To overcome these problems, we propose to carry out TTA by matching the feature distributions of test and training images, as modelled by a field-of-experts (FoE) prior. FoEs model complicated probability distributions as products of many simpler expert distributions. We use 1D marginal distributions of a trained task CNN's features as experts in the FoE model. Further, we compute principal components of patches of the task CNN's features, and consider the distributions of PCA loadings as additional experts. We validate the method on 5 MRI segmentation tasks (healthy tissues in 4 anatomical regions and lesions in 1 one anatomy), using data from 17 clinics, and on a MRI registration task, using data from 3 clinics. We find that the proposed FoE-based TTA is generically applicable in multiple tasks, and outperforms all previous TTA methods for lesion segmentation. For healthy tissue segmentation, the proposed method outperforms other task-agnostic methods, but a previous TTA method which is specifically designed for segmentation performs the best for most of the tested datasets. Our code is publicly available.