mlsanityData to 7 Oct 2026

On the Generalization Error of Meta Learning for the Gibbs Algorithm

arXiv 2304.14332 ↗First posted 27 Apr 2023cs.LG cs.IT math.ITcited by 2

Not on the map. Why?

Abstract (arXiv)

We analyze the generalization ability of joint-training meta learning algorithms via the Gibbs algorithm. Our exact characterization of the expected meta generalization error for the meta Gibbs algorithm is based on symmetrized KL information, which measures the dependence between all meta-training datasets and the output parameters, including task-specific and meta parameters. Additionally, we derive an exact characterization of the meta generalization error for the super-task Gibbs algorithm, in terms of conditional symmetrized KL information within the super-sample and super-task framework introduced in Steinke and Zakynthinou (2020) and Hellstrom and Durisi (2022) respectively. Our results also enable us to provide novel distribution-free generalization error upper bounds for these Gibbs algorithms applicable to meta learning.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 22 of its 44 references that are in the map. First posted on arXiv 27 Apr 2023.

Builds on

22 of 44 references.

References · 22

  1. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  2. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  3. 2018Deep Learning for Generic Object Detection: A Surveycited by 392
  4. 2017Information-theoretic analysis of generalization capability of learning algorithmscited by 299
  5. 2017Non-convex learning via Stochastic Gradient Langevin Dynamics: a nonasymptotic analysiscited by 242
  6. 2020Reasoning About Generalization via Conditional Mutual Informationcited by 145
  7. 2015On the properties of variational approximations of Gibbs posteriorscited by 121
  8. 2019Tightening Mutual Information Based Bounds on Generalization Errorcited by 117
  9. 2015How much does your data exploration overfit? Controlling bias via information usagecited by 95
  10. 2020PACOH: Bayes-Optimal Meta-Learning with PAC-Guaranteescited by 85
  11. 2017Meta-Learning by Adjusting Priors Based on Extended PAC-Bayes Theorycited by 76
  12. 2013A PAC-Bayesian bound for Lifelong Learningcited by 69
10 more
  1. 2019Continuous Meta-Learning without Taskscited by 50
  2. 2021Generalization Bounds For Meta-Learning: An Information-Theoretic Analysiscited by 45
  3. 2020Information-Theoretic Generalization Bounds for Meta-Learning and Applicationscited by 32
  4. 2019Distribution-Dependent Analysis of Gibbs-ERM Principlecited by 17
  5. 2021Characterizing and Understanding the Generalization Error of Transfer Learning with Gibbs Algorithmcited by 14
  6. 2021PAC-Bayes Bounds for Meta-learning with Data-Dependent Priorcited by 11
  7. 2022Evaluated CMI Bounds for Meta Learning: Tightness and Expressivenesscited by 11
  8. 2020A Distribution-Dependent Analysis of Meta-Learningcited by 6
  9. 2022How Does Pseudo-Labeling Affect the Generalization Error of the Semi-Supervised Gibbs Algorithm?cited by 5
  10. 2022A General framework for PAC-Bayes Bounds for Meta-Learningcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 0; 2026: 1.

2 papers.

  1. 2024How Does Distribution Matching Help Domain Generalization: An Information-theoretic Analysiscited by 3
  2. 2026Equivalence Between Nested Gibbs Measures and Log-Linear Combinations of Gibbs Measurescited by 0