# Cross-Domain Few-Shot Learning via Adaptive Transformer Networks

arXiv: https://arxiv.org/abs/2401.13987 · HTML: https://mlsanity.com/paper/2401.13987/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 37 references that are in the map.
- First posted on arXiv 25 Jan 2024.

## Builds on (most cited first, 30 of 32)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification](https://mlsanity.com/paper/1709.00029/) (2017)
- [Learning to Compare: Relation Network for Few-Shot Learning](https://mlsanity.com/paper/1711.06025/) (2017)
- [ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases](https://mlsanity.com/paper/1705.02315/) (2017)
- [Meta-Learning for Semi-Supervised Few-Shot Classification](https://mlsanity.com/paper/1803.00676/) (2018)
- [Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examples](https://mlsanity.com/paper/1903.03096/) (2019)
- [Meta-Transfer Learning for Few-Shot Learning](https://mlsanity.com/paper/1812.02391/) (2018)
- [Escaping the Big Data Paradigm with Compact Transformers](https://mlsanity.com/paper/2104.05704/) (2021)
- [Using Deep Learning for Image-Based Plant Disease Detection](https://mlsanity.com/paper/1604.03169/) (2016)
- [Cross-Domain Few-Shot Classification via Learned Feature-Wise Transformation](https://mlsanity.com/paper/2001.08735/) (2020)
- [CrossTransformers: spatially-aware few-shot transfer](https://mlsanity.com/paper/2007.11498/) (2020)
- [CDTrans: Cross-domain Transformer for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2109.06165/) (2021)
- [Pushing the Limits of Simple Pipelines for Few-Shot Learning: External Data and Fine-Tuning Make a Difference](https://mlsanity.com/paper/2204.07305/) (2022)
- [TVT: Transferable Vision Transformer for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2108.05988/) (2021)
- [Self-training for Few-shot Transfer Across Extreme Task Differences](https://mlsanity.com/paper/2010.07734/) (2020)
- [A Universal Representation Transformer Layer for Few-Shot Image Classification](https://mlsanity.com/paper/2006.11702/) (2020)
- [Cross-Domain Few-Shot Classification via Adversarial Task Augmentation](https://mlsanity.com/paper/2104.14385/) (2021)
- [StyleAdv: Meta Style Adversarial Training for Cross-Domain Few-Shot Learning](https://mlsanity.com/paper/2302.09309/) (2023)
- [Dynamic Distillation Network for Cross-Domain Few-Shot Recognition with Unlabeled Data](https://mlsanity.com/paper/2106.07807/) (2021)
- [Open Set Domain Adaptation: Theoretical Bound and Algorithm](https://mlsanity.com/paper/1907.08375/) (2019)
- [Self-Supervision Can Be a Good Few-Shot Learner](https://mlsanity.com/paper/2207.09176/) (2022)
- [Dual Adaptive Representation Alignment for Cross-domain Few-shot Learning](https://mlsanity.com/paper/2306.10511/) (2023)
- [Domain Adaptation via Bidirectional Cross-Attention Transformer](https://mlsanity.com/paper/2201.05887/) (2022)

## Built on by (most cited first, 7 of 7)

- [CyLens: Towards Reinventing Cyber Threat Intelligence in the Paradigm of Agentic Large Language Models](https://mlsanity.com/paper/2502.20791/) (2025)
- [Few-Shot Class Incremental Learning via Robust Transformer Approach](https://mlsanity.com/paper/2405.05984/) (2024)
- [Cross-Domain Few-Shot Learning with Coalescent Projections and Latent Space Reservation](https://mlsanity.com/paper/2507.15243/) (2025)
- [FreeGNN: Continual Source-Free Graph Neural Network Adaptation for Renewable Energy Forecasting](https://mlsanity.com/paper/2603.01657/) (2026)
- [A Cross-Domain Few-Shot Learning Method Based on Domain Knowledge Mapping](https://mlsanity.com/paper/2504.06608/) (2025)
- [ReCCur: A Recursive Corner-Case Curation Framework for Robust Vision-Language Understanding in Open and Edge Scenarios](https://mlsanity.com/paper/2601.03011/) (2026)
- [Cross-Domain Few-Shot Learning for Hyperspectral Image Classification Based on Mixup Foundation Model](https://mlsanity.com/paper/2601.22581/) (2026)

## Abstract (arXiv)

Most few-shot learning works rely on the same domain assumption between the base and the target tasks, hindering their practical applications. This paper proposes an adaptive transformer network (ADAPTER), a simple but effective solution for cross-domain few-shot learning where there exist large domain shifts between the base task and the target task. ADAPTER is built upon the idea of bidirectional cross-attention to learn transferable features between the two domains. The proposed architecture is trained with DINO to produce diverse, and less biased features to avoid the supervision collapse problem. Furthermore, the label smoothing approach is proposed to improve the consistency and reliability of the predictions by also considering the predicted labels of the close samples in the embedding space. The performance of ADAPTER is rigorously evaluated in the BSCD-FSL benchmarks in which it outperforms prior arts with significant margins.