# Meta-Transfer Learning for Few-Shot Learning

arXiv: https://arxiv.org/abs/1812.02391 · HTML: https://mlsanity.com/paper/1812.02391/

## Key facts

- Cited by 306 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Dec 2018.

## Built on by (most cited first, 30 of 306)

- [Deep Long-Tailed Learning: A Survey](https://mlsanity.com/paper/2110.04596/) (2021)
- [Few-Shot Incremental Learning with Continually Evolved Classifiers](https://mlsanity.com/paper/2104.03047/) (2021)
- [FSCE: Few-Shot Object Detection via Contrastive Proposal Encoding](https://mlsanity.com/paper/2103.05950/) (2021)
- [A Model or 603 Exemplars: Towards Memory-Efficient Class-Incremental Learning](https://mlsanity.com/paper/2205.13218/) (2022)
- [EcoTTA: Memory-Efficient Continual Test-time Adaptation via Self-distilled Regularization](https://mlsanity.com/paper/2303.01904/) (2023)
- [Overcoming Catastrophic Forgetting in Incremental Few-Shot Learning by Finding Flat Minima](https://mlsanity.com/paper/2111.01549/) (2021)
- [Image-to-Image Translation: Methods and Applications](https://mlsanity.com/paper/2101.08629/) (2021)
- [DeFRCN: Decoupled Faster R-CNN for Few-Shot Object Detection](https://mlsanity.com/paper/2108.09017/) (2021)
- [On the Role of Neural Collapse in Transfer Learning](https://mlsanity.com/paper/2112.15121/) (2021)
- [Learning from Few Examples: A Summary of Approaches to Few-Shot Learning](https://mlsanity.com/paper/2203.04291/) (2022)
- [Relational Embedding for Few-Shot Classification](https://mlsanity.com/paper/2108.09666/) (2021)
- [OTCE: A Transferability Metric for Cross-Domain Cross-Task Representations](https://mlsanity.com/paper/2103.13843/) (2021)
- [Few-Shot Lifelong Learning](https://mlsanity.com/paper/2103.00991/) (2021)
- [Transferability in Deep Learning: A Survey](https://mlsanity.com/paper/2201.05867/) (2022)
- [Exploring Complementary Strengths of Invariant and Equivariant Representations for Few-Shot Learning](https://mlsanity.com/paper/2103.01315/) (2021)
- [NatGen: Generative pre-training by "Naturalizing" source code](https://mlsanity.com/paper/2206.07585/) (2022)
- [Meta Balanced Network for Fair Face Recognition](https://mlsanity.com/paper/2205.06548/) (2022)
- [StyleAdv: Meta Style Adversarial Training for Cross-Domain Few-Shot Learning](https://mlsanity.com/paper/2302.09309/) (2023)
- [Class-Incremental Exemplar Compression for Class-Incremental Learning](https://mlsanity.com/paper/2303.14042/) (2023)
- [Shallow Bayesian Meta Learning for Real-World Few-Shot Recognition](https://mlsanity.com/paper/2101.02833/) (2021)
- [Learning from models beyond fine-tuning](https://mlsanity.com/paper/2310.08184/) (2023)
- [Deep Learning for HDR Imaging: State-of-the-Art and Future Trends](https://mlsanity.com/paper/2110.10394/) (2021)
- [Spatio-Temporal Graph Few-Shot Learning with Cross-City Knowledge Transfer](https://mlsanity.com/paper/2205.13947/) (2022)
- [Learning Dynamic Alignment via Meta-filter for Few-shot Learning](https://mlsanity.com/paper/2103.13582/) (2021)
- [Partner-Assisted Learning for Few-Shot Image Classification](https://mlsanity.com/paper/2109.07607/) (2021)
- [Sketch3T: Test-Time Training for Zero-Shot SBIR](https://mlsanity.com/paper/2203.14691/) (2022)
- [M3Net: Multi-view Encoding, Matching, and Fusion for Few-shot Fine-grained Action Recognition](https://mlsanity.com/paper/2308.03063/) (2023)
- [Few-shot Image Classification: Just Use a Library of Pre-trained Feature Extractors and a Simple Classifier](https://mlsanity.com/paper/2101.00562/) (2021)
- [SelfHAR: Improving Human Activity Recognition through Self-training with Unlabeled Data](https://mlsanity.com/paper/2102.06073/) (2021)
- [MetaHTR: Towards Writer-Adaptive Handwritten Text Recognition](https://mlsanity.com/paper/2104.01876/) (2021)

## Abstract (arXiv)

Meta-learning has been proposed as a framework to address the challenging few-shot learning setting. The key idea is to leverage a large number of similar few-shot tasks in order to learn how to adapt a base-learner to a new task for which only a few labeled samples are available. As deep neural networks (DNNs) tend to overfit using a few samples only, meta-learning typically uses shallow neural networks (SNNs), thus limiting its effectiveness. In this paper we propose a novel few-shot learning method called meta-transfer learning (MTL) which learns to adapt a deep NN for few shot learning tasks. Specifically, "meta" refers to training multiple tasks, and "transfer" is achieved by learning scaling and shifting functions of DNN weights for each task. In addition, we introduce the hard task (HT) meta-batch scheme as an effective learning curriculum for MTL. We conduct experiments using (5-class, 1-shot) and (5-class, 5-shot) recognition tasks on two challenging few-shot learning benchmarks: miniImageNet and Fewshot-CIFAR100. Extensive comparisons to related works validate that our meta-transfer learning approach trained with the proposed HT meta-batch scheme achieves top performance. An ablation study also shows that both components contribute to fast convergence and high accuracy.