mlsanityData to 14 Oct 2026

A General framework for PAC-Bayes Bounds for Meta-Learning

arXiv 2206.05454 ↗First posted 11 Jun 2022cs.LG stat.MLcited by 3

Not on the map. Why?

Abstract (arXiv)

Meta learning automatically infers an inductive bias, that includes the hyperparameter of the base-learning algorithm, by observing data from a finite number of related tasks. This paper studies PAC-Bayes bounds on meta generalization gap. The meta-generalization gap comprises two sources of generalization gaps: the environment-level and task-level gaps resulting from observation of a finite number of tasks and data samples per task, respectively. In this paper, by upper bounding arbitrary convex functions, which link the expected and empirical losses at the environment and also per-task levels, we obtain new PAC-Bayes bounds. Using these bounds, we develop new PAC-Bayes meta-learning algorithms. Numerical examples demonstrate the merits of the proposed novel bounds and algorithm in comparison to prior PAC-Bayes bounds for meta-learning.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 22 of its 42 references that are in the map. First posted on arXiv 11 Jun 2022.

Builds on

22 of 42 references.

References · 22

  1. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  2. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,101
  3. 2017Meta-SGD: Learning to Learn Quickly for Few-Shot Learningcited by 379
  4. 2017Information-theoretic analysis of generalization capability of learning algorithmscited by 303
  5. 2021User-friendly introduction to PAC-Bayes boundscited by 239
  6. 2019Adaptive Gradient-Based Meta-Learning Methodscited by 183
  7. 2019A Primer on PAC-Bayesian Learningcited by 132
  8. 2015On the properties of variational approximations of Gibbs posteriorscited by 121
  9. 2019Information-Theoretic Generalization Bounds for SGLD via Data-Dependent Estimatescited by 119
  10. 2019Tightening Mutual Information Based Bounds on Generalization Errorcited by 117
  11. 2020PACOH: Bayes-Optimal Meta-Learning with PAC-Guaranteescited by 85
  12. 2017Meta-Learning by Adjusting Priors Based on Extended PAC-Bayes Theorycited by 76
10 more
  1. 2013A PAC-Bayesian bound for Lifelong Learningcited by 69
  2. 2019Provable Guarantees for Gradient-Based Meta-Learningcited by 67
  3. 2020On the role of data in PAC-Bayes boundscited by 64
  4. 2020PAC-Bayes Analysis Beyond the Usual Boundscited by 63
  5. 2016A Strongly Quasiconvex PAC-Bayesian Boundcited by 54
  6. 2019Learning-to-Learn Stochastic Gradient Descent with Biased Regularizationcited by 49
  7. 2013A PAC-Bayesian Tutorial with A Dropout Boundcited by 43
  8. 2020Information-Theoretic Generalization Bounds for Meta-Learning and Applicationscited by 32
  9. 2019Still no free lunches: the price to pay for tighter PAC-Bayes boundscited by 14
  10. 2021PAC-Bayes Bounds for Meta-learning with Data-Dependent Priorcited by 11

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 1.

3 papers.

  1. 2024Position: Understanding LLMs Requires More Than Statistical Generalizationcited by 21
  2. 2023Metalearning with Very Few Samples Per Taskcited by 11
  3. 2023On the Generalization Error of Meta Learning for the Gibbs Algorithmcited by 2