# Few-Shot Adaptation of Pre-Trained Networks for Domain Shift

arXiv: https://arxiv.org/abs/2205.15234 · HTML: https://mlsanity.com/paper/2205.15234/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 45 references that are in the map.
- First posted on arXiv 30 May 2022.

## Builds on (most cited first, 30 of 32)

- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning](https://mlsanity.com/paper/1805.04687/) (2018)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [In Search of Lost Domain Generalization](https://mlsanity.com/paper/2007.01434/) (2020)
- [Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss](https://mlsanity.com/paper/1906.07413/) (2019)
- [Deeper, Broader and Artier Domain Generalization](https://mlsanity.com/paper/1710.03077/) (2017)
- [Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2002.08546/) (2020)
- [Playing for Data: Ground Truth from Computer Games](https://mlsanity.com/paper/1608.02192/) (2016)
- [Domain Generalization with MixStyle](https://mlsanity.com/paper/2104.02008/) (2021)
- [VisDA: The Visual Domain Adaptation Challenge](https://mlsanity.com/paper/1710.06924/) (2017)
- [Improving robustness against common corruptions by covariate shift adaptation](https://mlsanity.com/paper/2006.16971/) (2020)
- [Contrastive Adaptation Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1901.00976/) (2019)
- [Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net](https://mlsanity.com/paper/1807.09441/) (2018)
- [Explicit Inductive Bias for Transfer Learning with Convolutional Networks](https://mlsanity.com/paper/1802.01483/) (2018)
- [RobustNet: Improving Domain Generalization in Urban-Scene Segmentation via Instance Selective Whitening](https://mlsanity.com/paper/2103.15597/) (2021)
- [Domain-Specific Batch Normalization for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1906.03950/) (2019)
- [Generalized Source-free Domain Adaptation](https://mlsanity.com/paper/2108.01614/) (2021)
- [Source-free Domain Adaptation via Avatar Prototype Generation and Adaptation](https://mlsanity.com/paper/2106.15326/) (2021)
- [Switchable Whitening for Deep Representation Learning](https://mlsanity.com/paper/1904.09739/) (2019)
- [Iterative Normalization: Beyond Standardization towards Efficient Whitening](https://mlsanity.com/paper/1904.03441/) (2019)
- [Domain Impression: A Source Data Free Domain Adaptation Method](https://mlsanity.com/paper/2102.09003/) (2021)
- [Domain Adaptation without Source Data](https://mlsanity.com/paper/2007.01524/) (2020)
- [DELTA: DEep Learning Transfer using Feature Map with Attention for Convolutional Networks](https://mlsanity.com/paper/1901.09229/) (2019)
- [Batch-Instance Normalization for Adaptively Style-Invariant Neural Networks](https://mlsanity.com/paper/1805.07925/) (2018)
- [Learning a Universal Template for Few-shot Dataset Generalization](https://mlsanity.com/paper/2105.07029/) (2021)
- [Test-Time Training with Self-Supervision for Generalization under Distribution Shifts](https://mlsanity.com/paper/1909.13231/) (2019)
- [Contrastive Syn-to-Real Generalization](https://mlsanity.com/paper/2104.02290/) (2021)
- [Adapted Deep Embeddings: A Synthesis of Methods for k-Shot Inductive Transfer Learning](https://mlsanity.com/paper/1805.08402/) (2018)

## Built on by (most cited first, 16 of 16)

- [Semi-supervised Domain Adaptation in Graph Transfer Learning](https://mlsanity.com/paper/2309.10773/) (2023)
- [ALEX: Towards Effective Graph Transfer Learning with Noisy Labels](https://mlsanity.com/paper/2309.14673/) (2023)
- [Few-shot Fine-tuning is All You Need for Source-free Domain Adaptation](https://mlsanity.com/paper/2304.00792/) (2023)
- [DUET: Optimizing Training Data Mixtures via Feedback from Unseen Evaluation Tasks](https://mlsanity.com/paper/2502.00270/) (2025)
- [Enhancing Test Time Adaptation with Few-shot Guidance](https://mlsanity.com/paper/2409.01341/) (2024)
- [Optimal Transport-Guided Source-Free Adaptation for Face Anti-Spoofing](https://mlsanity.com/paper/2503.22984/) (2025)
- [Analytical Survey of Learning with Low-Resource Data: From Analysis to Investigation](https://mlsanity.com/paper/2510.08962/) (2025)
- [A Survey of Deep Visual Cross-Domain Few-Shot Learning](https://mlsanity.com/paper/2303.09253/) (2023)
- [Few-shot Adaptation to Distribution Shifts By Mixing Source and Target Embeddings](https://mlsanity.com/paper/2305.14521/) (2023)
- [AdaMesh: Personalized Facial Expressions and Head Poses for Adaptive Speech-Driven 3D Facial Animation](https://mlsanity.com/paper/2310.07236/) (2023)
- [Benchmarking Low-Shot Robustness to Natural Distribution Shifts](https://mlsanity.com/paper/2304.11263/) (2023)
- [Efficient Cross-Domain Federated Learning by MixStyle Approximation](https://mlsanity.com/paper/2312.07064/) (2023)
- [AdaTreeFormer: Few Shot Domain Adaptation for Tree Counting from a Single High-Resolution Image](https://mlsanity.com/paper/2402.02956/) (2024)
- [SelectNAdapt: Support Set Selection for Few-Shot Domain Adaptation](https://mlsanity.com/paper/2308.04946/) (2023)
- [Meta-TTT: A Meta-learning Minimax Framework For Test-Time Training](https://mlsanity.com/paper/2410.01709/) (2024)
- [Domain Borders Are There to Be Crossed With Federated Few-Shot Adaptation](https://mlsanity.com/paper/2507.10160/) (2025)

## Abstract (arXiv)

Deep networks are prone to performance degradation when there is a domain shift between the source (training) data and target (test) data. Recent test-time adaptation methods update batch normalization layers of pre-trained source models deployed in new target environments with streaming data to mitigate such performance degradation. Although such methods can adapt on-the-fly without first collecting a large target domain dataset, their performance is dependent on streaming conditions such as mini-batch size and class-distribution, which can be unpredictable in practice. In this work, we propose a framework for few-shot domain adaptation to address the practical challenges of data-efficient adaptation. Specifically, we propose a constrained optimization of feature normalization statistics in pre-trained source models supervised by a small support set from the target domain. Our method is easy to implement and improves source model performance with as few as one sample per class for classification tasks. Extensive experiments on 5 cross-domain classification and 4 semantic segmentation datasets show that our method achieves more accurate and reliable performance than test-time adaptation, while not being constrained by streaming conditions.