# Learning What and Where to Transfer

arXiv: https://arxiv.org/abs/1905.05901 · HTML: https://mlsanity.com/paper/1905.05901/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 May 2019.

## Built on by (most cited first, 30 of 43)

- [Show, Attend and Distill:Knowledge Distillation via Attention-based Feature Matching](https://mlsanity.com/paper/2102.02973/) (2021)
- [DOC2PPT: Automatic Presentation Slides Generation from Scientific Documents](https://mlsanity.com/paper/2101.11796/) (2021)
- [Transferability in Deep Learning: A Survey](https://mlsanity.com/paper/2201.05867/) (2022)
- [Transfer without Forgetting](https://mlsanity.com/paper/2206.00388/) (2022)
- [AlphaNet: Improved Training of Supernets with Alpha-Divergence](https://mlsanity.com/paper/2102.07954/) (2021)
- [A Data-Based Perspective on Transfer Learning](https://mlsanity.com/paper/2207.05739/) (2022)
- [Do Multiple Instance Learning Models Transfer?](https://mlsanity.com/paper/2506.09022/) (2025)
- [Learngene: Inheriting Condensed Knowledge from the Ancestry Model to Descendant Models](https://mlsanity.com/paper/2305.02279/) (2023)
- [Distilling a Powerful Student Model via Online Knowledge Distillation](https://mlsanity.com/paper/2103.14473/) (2021)
- [Categories of Response-Based, Feature-Based, and Relation-Based Knowledge Distillation](https://mlsanity.com/paper/2306.10687/) (2023)
- [Generic-to-Specific Distillation of Masked Autoencoders](https://mlsanity.com/paper/2302.14771/) (2023)
- [Adaptive Multi-Teacher Knowledge Distillation with Meta-Learning](https://mlsanity.com/paper/2306.06634/) (2023)
- [Exploiting User Comments for Early Detection of Fake News Prior to Users' Commenting](https://mlsanity.com/paper/2310.10429/) (2023)
- [Large-Scale Meta-Learning with Continual Trajectory Shifting](https://mlsanity.com/paper/2102.07215/) (2021)
- [Meta-Learning with Self-Improving Momentum Target](https://mlsanity.com/paper/2210.05185/) (2022)
- [Doing More with Less: Overcoming Data Scarcity for POI Recommendation via Cross-Region Transfer](https://mlsanity.com/paper/2201.06095/) (2022)
- [DistPro: Searching A Fast Knowledge Distillation Process via Meta Optimization](https://mlsanity.com/paper/2204.05547/) (2022)
- [Dual-path Frequency Discriminators for Few-shot Anomaly Detection](https://mlsanity.com/paper/2403.04151/) (2024)
- [Transferring Knowledge from Large Foundation Models to Small Downstream Models](https://mlsanity.com/paper/2406.07337/) (2024)
- [Self-supervised Auxiliary Learning for Graph Neural Networks via Meta-Learning](https://mlsanity.com/paper/2103.00771/) (2021)
- [Meta Auxiliary Learning for Facial Action Unit Detection](https://mlsanity.com/paper/2105.06620/) (2021)
- [Temporal Transfer Learning for Traffic Optimization with Coarse-grained Advisory Autonomy](https://mlsanity.com/paper/2312.09436/) (2023)
- [Few-Shot Learning of Compact Models via Task-Specific Meta Distillation](https://mlsanity.com/paper/2210.09922/) (2022)
- [Aggregate or Not? Exploring Where to Privatize in DNN Based Federated Learning Under Different Non-IID Scenes](https://mlsanity.com/paper/2107.11954/) (2021)
- [DAC-MR: Data Augmentation Consistency Based Meta-Regularization for Meta-Learning](https://mlsanity.com/paper/2305.07892/) (2023)
- [Leveraging Angular Distributions for Improved Knowledge Distillation](https://mlsanity.com/paper/2302.14130/) (2023)
- [Token Coordinated Prompt Attention is Needed for Visual Prompting](https://mlsanity.com/paper/2505.02406/) (2025)
- [Auto-Transfer: Learning to Route Transferrable Representations](https://mlsanity.com/paper/2202.01011/) (2022)
- [Understanding Optimal Feature Transfer via a Fine-Grained Bias-Variance Analysis](https://mlsanity.com/paper/2404.12481/) (2024)
- [Efficient Transfer Learning via Joint Adaptation of Network Architecture and Weight](https://mlsanity.com/paper/2105.08994/) (2021)

## Abstract (arXiv)

As the application of deep learning has expanded to real-world problems with insufficient volume of training data, transfer learning recently has gained much attention as means of improving the performance in such small-data regime. However, when existing methods are applied between heterogeneous architectures and tasks, it becomes more important to manage their detailed configurations and often requires exhaustive tuning on them for the desired performance. To address the issue, we propose a novel transfer learning approach based on meta-learning that can automatically learn what knowledge to transfer from the source network to where in the target network. Given source and target networks, we propose an efficient training scheme to learn meta-networks that decide (a) which pairs of layers between the source and target networks should be matched for knowledge transfer and (b) which features and how much knowledge from each feature should be transferred. We validate our meta-transfer approach against recent transfer learning methods on various datasets and network architectures, on which our automated scheme significantly outperforms the prior baselines that find "what and where to transfer" in a hand-crafted manner.