# Understanding Cross-Domain Few-Shot Learning Based on Domain Similarity and Few-Shot Difficulty

arXiv: https://arxiv.org/abs/2202.01339 · HTML: https://mlsanity.com/paper/2202.01339/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Builds on 59 of its 86 references that are in the map.
- First posted on arXiv 1 Feb 2022.

## Builds on (most cited first, 30 of 59)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence](https://mlsanity.com/paper/2001.07685/) (2020)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification](https://mlsanity.com/paper/1709.00029/) (2017)
- [Context Encoders: Feature Learning by Inpainting](https://mlsanity.com/paper/1604.07379/) (2016)
- [Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere](https://mlsanity.com/paper/2005.10242/) (2020)
- [Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1603.09246/) (2016)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [EfficientNetV2: Smaller Models and Faster Training](https://mlsanity.com/paper/2104.00298/) (2021)
- [Colorful Image Colorization](https://mlsanity.com/paper/1603.08511/) (2016)
- [Unsupervised Visual Representation Learning by Context Prediction](https://mlsanity.com/paper/1505.05192/) (2015)
- [ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases](https://mlsanity.com/paper/1705.02315/) (2017)

## Built on by (most cited first, 30 of 37)

- [Closed-form merging of parameter-efficient modules for Federated Continual Learning](https://mlsanity.com/paper/2410.17961/) (2024)
- [Context-Aware Meta-Learning](https://mlsanity.com/paper/2310.10971/) (2023)
- [Semantic Residual Prompts for Continual Learning](https://mlsanity.com/paper/2403.06870/) (2024)
- [A Second-Order Perspective on Model Compositionality and Incremental Learning](https://mlsanity.com/paper/2405.16350/) (2024)
- [Compositional Few-Shot Class-Incremental Learning](https://mlsanity.com/paper/2405.17022/) (2024)
- [CLIP with Generative Latent Replay: a Strong Baseline for Incremental Learning](https://mlsanity.com/paper/2407.15793/) (2024)
- [Exploring Cross-Domain Few-Shot Classification via Frequency-Aware Prompting](https://mlsanity.com/paper/2406.16422/) (2024)
- [VT-FSL: Bridging Vision and Text with LLMs for Few-Shot Learning](https://mlsanity.com/paper/2509.25033/) (2025)
- [Meta Transfer of Self-Supervised Knowledge: Foundation Model in Action for Post-Traumatic Epilepsy Prediction](https://mlsanity.com/paper/2312.14204/) (2023)
- [MetaCoCo: A New Few-Shot Classification Benchmark with Spurious Correlation](https://mlsanity.com/paper/2404.19644/) (2024)
- [Towards Few-Shot Learning in the Open World: A Review and Beyond](https://mlsanity.com/paper/2408.09722/) (2024)
- [Analytical Survey of Learning with Low-Resource Data: From Analysis to Investigation](https://mlsanity.com/paper/2510.08962/) (2025)
- [Task-Distributionally Robust Data-Free Meta-Learning](https://mlsanity.com/paper/2311.14756/) (2023)
- [Random Registers for Cross-Domain Few-Shot Learning](https://mlsanity.com/paper/2506.02843/) (2025)
- [A Survey of Deep Visual Cross-Domain Few-Shot Learning](https://mlsanity.com/paper/2303.09253/) (2023)
- [Leveraging Normalization Layer in Adapters With Progressive Learning and Adaptive Distillation for Cross-Domain Few-Shot Learning](https://mlsanity.com/paper/2312.11260/) (2023)
- [Revisiting Continuity of Image Tokens for Cross-domain Few-shot Learning](https://mlsanity.com/paper/2506.03110/) (2025)
- [HyperFlow: Gradient-Free Emulation of Few-Shot Fine-Tuning](https://mlsanity.com/paper/2504.15323/) (2025)
- [Exploring Pre-training Across Domains for Few-Shot Surgical Skill Assessment](https://mlsanity.com/paper/2509.09327/) (2025)
- [DVLA-RL: Dual-Level Vision-Language Alignment with Reinforcement Learning Gating for Few-Shot Learning](https://mlsanity.com/paper/2602.00795/) (2026)
- [Addressing Exacerbated Attention Sink for Source-Free Cross-Domain Few-Shot Learning](https://mlsanity.com/paper/2605.25799/) (2026)
- [Generalized Cross-domain Multi-label Few-shot Learning for Chest X-rays](https://mlsanity.com/paper/2309.04462/) (2023)
- [Exploring Active Learning in Meta-Learning: Enhancing Context Set Labeling](https://mlsanity.com/paper/2311.02879/) (2023)
- [Chem-FINESE: Validating Fine-Grained Few-shot Entity Extraction through Text Reconstruction](https://mlsanity.com/paper/2401.10189/) (2024)
- [Task Attribute Distance for Few-Shot Learning: Theoretical Analysis and Applications](https://mlsanity.com/paper/2403.03535/) (2024)
- [Generalizable Representation Learning for fMRI-based Neurological Disorder Identification](https://mlsanity.com/paper/2412.16197/) (2024)
- [Universal Algorithm-Implicit Learning](https://mlsanity.com/paper/2602.14761/) (2026)
- [Improving CLIP Adaptation by Breaking Tail Alignment for Source-Free Cross-Domain Few-Shot Learning](https://mlsanity.com/paper/2605.29776/) (2026)
- [Learning Transferable Adversarial Robust Representations via Multi-view Consistency](https://mlsanity.com/paper/2210.10485/) (2022)
- [Few and Fewer: Learning Better from Few Examples Using Fewer Base Classes](https://mlsanity.com/paper/2401.15834/) (2024)

## Abstract (arXiv)

Cross-domain few-shot learning (CD-FSL) has drawn increasing attention for handling large differences between the source and target domains--an important concern in real-world scenarios. To overcome these large differences, recent works have considered exploiting small-scale unlabeled data from the target domain during the pre-training stage. This data enables self-supervised pre-training on the target domain, in addition to supervised pre-training on the source domain. In this paper, we empirically investigate which pre-training is preferred based on domain similarity and few-shot difficulty of the target domain. We discover that the performance gain of self-supervised pre-training over supervised pre-training becomes large when the target domain is dissimilar to the source domain, or the target domain itself has low few-shot difficulty. We further design two pre-training schemes, mixed-supervised and two-stage learning, that improve performance. In this light, we present six findings for CD-FSL, which are supported by extensive experiments and analyses on three source and eight target benchmark datasets with varying levels of domain similarity and few-shot difficulty. Our code is available at https://github.com/sungnyun/understanding-cdfsl.