# Transfer Meta-Learning: Information-Theoretic Bounds and Information Meta-Risk Minimization

arXiv: https://arxiv.org/abs/2011.02872 · HTML: https://mlsanity.com/paper/2011.02872/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Nov 2020.

## Built on by (most cited first, 14 of 14)

- [User-friendly introduction to PAC-Bayes bounds](https://mlsanity.com/paper/2110.11216/) (2021)
- [Information-Theoretic Analysis of Unsupervised Domain Adaptation](https://mlsanity.com/paper/2210.00706/) (2022)
- [Modeling Human Exploration Through Resource-Rational Reinforcement Learning](https://mlsanity.com/paper/2201.11817/) (2022)
- [Is Bayesian Model-Agnostic Meta Learning Better than Model-Agnostic Meta Learning, Provably?](https://mlsanity.com/paper/2203.03059/) (2022)
- [Evaluated CMI Bounds for Meta Learning: Tightness and Expressiveness](https://mlsanity.com/paper/2210.06511/) (2022)
- [An Information-Theoretic Analysis of the Impact of Task Similarity on Meta-Learning](https://mlsanity.com/paper/2101.08390/) (2021)
- [Learning an Explicit Hyperparameter Prediction Function Conditioned on Tasks](https://mlsanity.com/paper/2107.02378/) (2021)
- [Bayes meets Bernstein at the Meta Level: an Analysis of Fast Rates in Meta-Learning with PAC-Bayes](https://mlsanity.com/paper/2302.11709/) (2023)
- [PAC-Bayes Bounds for Bandit Problems: A Survey and Experimental Comparison](https://mlsanity.com/paper/2211.16110/) (2022)
- [Meta-strategy for Learning Tuning Parameters with Guarantees](https://mlsanity.com/paper/2102.02504/) (2021)
- [Information-Theoretic Generalization Bounds for Transductive Learning and its Applications](https://mlsanity.com/paper/2311.04561/) (2023)
- [Transfer Bayesian Meta-learning via Weighted Free Energy Minimization](https://mlsanity.com/paper/2106.10711/) (2021)
- [Online Bayesian Meta-Learning for Cognitive Tracking Radar](https://mlsanity.com/paper/2207.06917/) (2022)
- [Generalization Theory for Through-the-Wall Radar Human Activity Recognition](https://mlsanity.com/paper/2607.08144/) (2026)

## Abstract (arXiv)

Meta-learning automatically infers an inductive bias by observing data from a number of related tasks. The inductive bias is encoded by hyperparameters that determine aspects of the model class or training algorithm, such as initialization or learning rate. Meta-learning assumes that the learning tasks belong to a task environment, and that tasks are drawn from the same task environment both during meta-training and meta-testing. This, however, may not hold true in practice. In this paper, we introduce the problem of transfer meta-learning, in which tasks are drawn from a target task environment during meta-testing that may differ from the source task environment observed during meta-training. Novel information-theoretic upper bounds are obtained on the transfer meta-generalization gap, which measures the difference between the meta-training loss, available at the meta-learner, and the average loss on meta-test data from a new, randomly selected, task in the target task environment. The first bound, on the average transfer meta-generalization gap, captures the meta-environment shift between source and target task environments via the KL divergence between source and target data distributions. The second, PAC-Bayesian bound, and the third, single-draw bound, account for this shift via the log-likelihood ratio between source and target task distributions. Furthermore, two transfer meta-learning solutions are introduced. For the first, termed Empirical Meta-Risk Minimization (EMRM), we derive bounds on the average optimality gap. The second, referred to as Information Meta-Risk Minimization (IMRM), is obtained by minimizing the PAC-Bayesian bound. IMRM is shown via experiments to potentially outperform EMRM.