# Source-Free Adaptation to Measurement Shift via Bottom-Up Feature Restoration

arXiv: https://arxiv.org/abs/2107.05446 · HTML: https://mlsanity.com/paper/2107.05446/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 82 references that are in the map.
- First posted on arXiv 12 Jul 2021.

## Builds on (most cited first, 30 of 43)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [Do ImageNet Classifiers Generalize to ImageNet?](https://mlsanity.com/paper/1902.10811/) (2019)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [Learning Transferable Features with Deep Adaptation Networks](https://mlsanity.com/paper/1502.02791/) (2015)
- [Adversarial Discriminative Domain Adaptation](https://mlsanity.com/paper/1702.05464/) (2017)
- [How transferable are features in deep neural networks?](https://mlsanity.com/paper/1411.1792/) (2014)
- [Deep CORAL: Correlation Alignment for Deep Domain Adaptation](https://mlsanity.com/paper/1607.01719/) (2016)
- [In Search of Lost Domain Generalization](https://mlsanity.com/paper/2007.01434/) (2020)
- [Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2002.08546/) (2020)
- [Causal inference using invariant prediction: identification and confidence intervals](https://mlsanity.com/paper/1501.01332/) (2015)
- [Improving robustness against common corruptions by covariate shift adaptation](https://mlsanity.com/paper/2006.16971/) (2020)
- [Confounding variables can degrade generalization performance of radiological deep learning models](https://mlsanity.com/paper/1807.00431/) (2018)
- [Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Coming](https://mlsanity.com/paper/1907.07484/) (2019)
- [Revisiting Batch Normalization For Practical Domain Adaptation](https://mlsanity.com/paper/1603.04779/) (2016)
- [Generalisation in humans and deep neural networks](https://mlsanity.com/paper/1808.08750/) (2018)
- [Evaluating Prediction-Time Batch Normalization for Robustness under Covariate Shift](https://mlsanity.com/paper/2006.10963/) (2020)
- [Universal Source-Free Domain Adaptation](https://mlsanity.com/paper/2004.04393/) (2020)
- [A DIRT-T Approach to Unsupervised Domain Adaptation](https://mlsanity.com/paper/1802.08735/) (2018)
- [EMNIST: an extension of MNIST to handwritten letters](https://mlsanity.com/paper/1702.05373/) (2017)
- [Central Moment Discrepancy (CMD) for Domain-Invariant Representation Learning](https://mlsanity.com/paper/1702.08811/) (2017)

## Built on by (most cited first, 30 of 50)

- [A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts](https://mlsanity.com/paper/2303.15361/) (2023)
- [Surgical Fine-Tuning Improves Adaptation to Distribution Shifts](https://mlsanity.com/paper/2210.11466/) (2022)
- [Source-Free Unsupervised Domain Adaptation: A Survey](https://mlsanity.com/paper/2301.00265/) (2022)
- [Test-Time Model Adaptation with Only Forward Passes](https://mlsanity.com/paper/2404.01650/) (2024)
- [Real-Time Style Modelling of Human Locomotion via Feature-Wise Transformations and Local Motion Phases](https://mlsanity.com/paper/2201.04439/) (2022)
- [Back to the Source: Diffusion-Driven Test-Time Adaptation](https://mlsanity.com/paper/2207.03442/) (2022)
- [Probable Domain Generalization via Quantile Risk Minimization](https://mlsanity.com/paper/2207.09944/) (2022)
- [Test-Time Adaptation via Self-Training with Nearest Neighbor Information](https://mlsanity.com/paper/2207.10792/) (2022)
- [On Pitfalls of Test-Time Adaptation](https://mlsanity.com/paper/2306.03536/) (2023)
- [Revisiting Realistic Test-Time Training: Sequential Inference and Adaptation by Anchored Clustering](https://mlsanity.com/paper/2206.02721/) (2022)
- [ActMAD: Activation Matching to Align Distributions for Test-Time-Training](https://mlsanity.com/paper/2211.12870/) (2022)
- [Data-Free Knowledge Transfer: A Survey](https://mlsanity.com/paper/2112.15278/) (2021)
- [Joint Learning of Label and Environment Causal Independence for Graph Out-of-Distribution Generalization](https://mlsanity.com/paper/2306.01103/) (2023)
- [Generation, augmentation, and alignment: A pseudo-source domain based method for source-free domain adaptation](https://mlsanity.com/paper/2109.04015/) (2021)
- [TeSLA: Test-Time Self-Learning With Automatic Adversarial Augmentation](https://mlsanity.com/paper/2303.09870/) (2023)
- [CAFA: Class-Aware Feature Alignment for Test-Time Adaptation](https://mlsanity.com/paper/2206.00205/) (2022)
- [Spuriosity Didn't Kill the Classifier: Using Invariant Predictions to Harness Spurious Features](https://mlsanity.com/paper/2307.09933/) (2023)
- [Multi-Modal Continual Test-Time Adaptation for 3D Semantic Segmentation](https://mlsanity.com/paper/2303.10457/) (2023)
- [ProSFDA: Prompt Learning based Source-free Domain Adaptation for Medical Image Segmentation](https://mlsanity.com/paper/2211.11514/) (2022)
- [Active Test-Time Adaptation: Theoretical Analyses and An Algorithm](https://mlsanity.com/paper/2404.05094/) (2024)
- [Consistency Regularization for Generalizable Source-free Domain Adaptation](https://mlsanity.com/paper/2308.01587/) (2023)
- [Partial Identifiability for Domain Adaptation](https://mlsanity.com/paper/2306.06510/) (2023)
- [Distribution Alignment for Fully Test-Time Adaptation with Dynamic Online Data Streams](https://mlsanity.com/paper/2407.12128/) (2024)
- [Towards Data-Free Domain Generalization](https://mlsanity.com/paper/2110.04545/) (2021)
- [VPA: Fully Test-Time Visual Prompt Adaptation](https://mlsanity.com/paper/2309.15251/) (2023)
- [Test-time Adaptation for Regression by Subspace Alignment](https://mlsanity.com/paper/2410.03263/) (2024)
- [Towards Understanding GD with Hard and Conjugate Pseudo-labels for Test-Time Adaptation](https://mlsanity.com/paper/2210.10019/) (2022)
- [Trust your Good Friends: Source-free Domain Adaptation by Reciprocal Neighborhood Clustering](https://mlsanity.com/paper/2309.00528/) (2023)
- [DALLMi: Domain Adaption for LLM-based Multi-label Classifier](https://mlsanity.com/paper/2405.01883/) (2024)
- [Uncertainty-Guided Mixup for Semi-Supervised Domain Adaptation without Source Data](https://mlsanity.com/paper/2107.06707/) (2021)

## Abstract (arXiv)

Source-free domain adaptation (SFDA) aims to adapt a model trained on labelled data in a source domain to unlabelled data in a target domain without access to the source-domain data during adaptation. Existing methods for SFDA leverage entropy-minimization techniques which: (i) apply only to classification; (ii) destroy model calibration; and (iii) rely on the source model achieving a good level of feature-space class-separation in the target domain. We address these issues for a particularly pervasive type of domain shift called measurement shift which can be resolved by restoring the source features rather than extracting new ones. In particular, we propose Feature Restoration (FR) wherein we: (i) store a lightweight and flexible approximation of the feature distribution under the source data; and (ii) adapt the feature-extractor such that the approximate feature distribution under the target data realigns with that saved on the source. We additionally propose a bottom-up training scheme which boosts performance, which we call Bottom-Up Feature Restoration (BUFR). On real and synthetic data, we demonstrate that BUFR outperforms existing SFDA methods in terms of accuracy, calibration, and data efficiency, while being less reliant on the performance of the source model in the target domain.