# Self-training for Few-shot Transfer Across Extreme Task Differences

arXiv: https://arxiv.org/abs/2010.07734 · HTML: https://mlsanity.com/paper/2010.07734/

## Key facts

- Cited by 76 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Oct 2020.

## Built on by (most cited first, 30 of 76)

- [A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities](https://mlsanity.com/paper/2205.06743/) (2022)
- [Pushing the Limits of Simple Pipelines for Few-Shot Learning: External Data and Fine-Tuning Make a Difference](https://mlsanity.com/paper/2204.07305/) (2022)
- [StyleAdv: Meta Style Adversarial Training for Cross-Domain Few-Shot Learning](https://mlsanity.com/paper/2302.09309/) (2023)
- [Dynamic Distillation Network for Cross-Domain Few-Shot Recognition with Unlabeled Data](https://mlsanity.com/paper/2106.07807/) (2021)
- [Meta-FDMixup: Cross-Domain Few-Shot Learning Guided by Labeled Target Data](https://mlsanity.com/paper/2107.11978/) (2021)
- [Learning to Detect Mobile Objects from LiDAR Scans Without Labels](https://mlsanity.com/paper/2203.15882/) (2022)
- [Understanding Cross-Domain Few-Shot Learning Based on Domain Similarity and Few-Shot Difficulty](https://mlsanity.com/paper/2202.01339/) (2022)
- [Ranking Distance Calibration for Cross-Domain Few-Shot Learning](https://mlsanity.com/paper/2112.00260/) (2021)
- [Wave-SAN: Wavelet based Style Augmentation Network for Cross-Domain Few-Shot Learning](https://mlsanity.com/paper/2203.07656/) (2022)
- [Context-Aware Meta-Learning](https://mlsanity.com/paper/2310.10971/) (2023)
- [FHIST: A Benchmark for Few-shot Classification of Histological Images](https://mlsanity.com/paper/2206.00092/) (2022)
- [Self-Supervision Can Be a Good Few-Shot Learner](https://mlsanity.com/paper/2207.09176/) (2022)
- [CD-FSOD: A Benchmark for Cross-domain Few-shot Object Detection](https://mlsanity.com/paper/2210.05311/) (2022)
- [Comparing Transfer and Meta Learning Approaches on a Unified Few-Shot Classification Benchmark](https://mlsanity.com/paper/2104.02638/) (2021)
- [TGDM: Target Guided Dynamic Mixup for Cross-Domain Few-Shot Learning](https://mlsanity.com/paper/2210.05392/) (2022)
- [A Closer Look at Prototype Classifier for Few-shot Image Classification](https://mlsanity.com/paper/2110.05076/) (2021)
- [POODLE: Improving Few-shot Learning via Penalizing Out-of-Distribution Samples](https://mlsanity.com/paper/2206.04679/) (2022)
- [Dual Adaptive Representation Alignment for Cross-domain Few-shot Learning](https://mlsanity.com/paper/2306.10511/) (2023)
- [On the Importance of Distractors for Few-Shot Classification](https://mlsanity.com/paper/2109.09883/) (2021)
- [Hierarchical Variational Memory for Few-shot Learning Across Domains](https://mlsanity.com/paper/2112.08181/) (2021)
- [Deep Learning for Cross-Domain Few-Shot Visual Recognition: A Survey](https://mlsanity.com/paper/2303.08557/) (2023)
- [Multimodal Cross-Domain Few-Shot Learning for Egocentric Action Recognition](https://mlsanity.com/paper/2405.19917/) (2024)
- [CDFSL-V: Cross-Domain Few-Shot Learning for Videos](https://mlsanity.com/paper/2309.03989/) (2023)
- [Enhancing Information Maximization with Distance-Aware Contrastive Learning for Source-Free Cross-Domain Few-Shot Learning](https://mlsanity.com/paper/2403.01966/) (2024)
- [Layout Agnostic Human Activity Recognition in Smart Homes through Textual Descriptions Of Sensor Triggers (TDOST)](https://mlsanity.com/paper/2405.12368/) (2024)
- [Prompt as Free Lunch: Enhancing Diversity in Source-Free Cross-domain Few-shot Learning through Semantic-Guided Prompting](https://mlsanity.com/paper/2412.00767/) (2024)
- [ME-D2N: Multi-Expert Domain Decompositional Network for Cross-Domain Few-Shot Learning](https://mlsanity.com/paper/2210.05280/) (2022)
- [Cross-Domain HAR: Few Shot Transfer Learning for Human Activity Recognition](https://mlsanity.com/paper/2310.14390/) (2023)
- [Exploring Cross-Domain Few-Shot Classification via Frequency-Aware Prompting](https://mlsanity.com/paper/2406.16422/) (2024)
- [On the Importance of Firth Bias Reduction in Few-Shot Classification](https://mlsanity.com/paper/2110.02529/) (2021)

## Abstract (arXiv)

Most few-shot learning techniques are pre-trained on a large, labeled "base dataset". In problem domains where such large labeled datasets are not available for pre-training (e.g., X-ray, satellite images), one must resort to pre-training in a different "source" problem domain (e.g., ImageNet), which can be very different from the desired target task. Traditional few-shot and transfer learning techniques fail in the presence of such extreme differences between the source and target tasks. In this paper, we present a simple and effective solution to tackle this extreme domain gap: self-training a source domain representation on unlabeled data from the target domain. We show that this improves one-shot performance on the target domain by 2.9 points on average on the challenging BSCD-FSL benchmark consisting of datasets from multiple domains. Our code is available at https://github.com/cpphoo/STARTUP.