mlsanityData to 7 Oct 2026

PAC-Bayes Bounds for Meta-learning with Data-Dependent Prior

arXiv 2102.03748 ↗First posted 7 Feb 2021cs.LG stat.MLcited by 11

Not on the map. Why?

Abstract (arXiv)

By leveraging experience from previous tasks, meta-learning algorithms can achieve effective fast adaptation ability when encountering new tasks. However it is unclear how the generalization property applies to new tasks. Probably approximately correct (PAC) Bayes bound theory provides a theoretical framework to analyze the generalization performance for meta-learning. We derive three novel generalisation error bounds for meta-learning based on PAC-Bayes relative entropy bound. Furthermore, using the empirical risk minimization (ERM) method, a PAC-Bayes bound for meta-learning with data-dependent prior is developed. Experiments illustrate that the proposed three PAC-Bayes bounds for meta-learning guarantee a competitive generalization performance guarantee, and the extended PAC-Bayes bound with data-dependent prior can achieve rapid convergence ability.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 31 of its 51 references that are in the map. First posted on arXiv 7 Feb 2021.

Builds on

31 of 51 references.

References · 31

  1. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  2. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  3. 2016Matching Networks for One Shot Learningcited by 2,246
  4. 2016Learning to learn by gradient descent by gradient descentcited by 616
  5. 2017Exploring Generalization in Deep Learningcited by 595
  6. 2017Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Datacited by 527
  7. 2018Meta-Learning with Latent Embedding Optimizationcited by 476
  8. 2017A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networkscited by 309
  9. 2018How to train your MAMLcited by 306
  10. 2018Probabilistic Model-Agnostic Meta-Learningcited by 254
  11. 2018Bayesian Model-Agnostic Meta-Learningcited by 194
  12. 2018Recasting Gradient-Based Meta-Learning as Hierarchical Bayescited by 180
19 more
  1. 2018Non-Vacuous Generalization Bounds at the ImageNet Scale: A PAC-Bayesian Compression Approachcited by 125
  2. 2015On the properties of variational approximations of Gibbs posteriorscited by 121
  3. 2020Tighter risk certificates for neural networkscited by 112
  4. 2019Meta-Learning without Memorizationcited by 92
  5. 2016PAC-Bayesian Theory Meets Bayesian Inferencecited by 87
  6. 2017Entropy-SGD optimizes the prior of a PAC-Bayes bound: Generalization properties of Entropy-SGD and data-dependent priorscited by 81
  7. 2017Meta-Learning by Adjusting Priors Based on Extended PAC-Bayes Theorycited by 76
  8. 2013A PAC-Bayesian bound for Lifelong Learningcited by 69
  9. 2020On the role of data in PAC-Bayes boundscited by 64
  10. 2016A Strongly Quasiconvex PAC-Bayesian Boundcited by 54
  11. 2019Deterministic PAC-Bayesian generalization bounds for deep networks via generalizing noise-resiliencecited by 51
  12. 2019Learning to Balance: Bayesian Meta-Learning for Imbalanced and Out-of-distribution Taskscited by 48
  13. 2020PAC-Bayes unleashed: generalisation bounds with unbounded lossescited by 43
  14. 2017A PAC-Bayesian Analysis of Randomized Learning with Application to Stochastic Gradient Descentcited by 29
  15. 2019Alpha MAML: Adaptive Model-Agnostic Meta-Learningcited by 25
  16. 2018PAC-Bayes bounds for stable algorithms with instance-dependent priorscited by 21
  17. 2019PAC-Bayes under potentially heavy tailscited by 16
  18. 2019PAC-Bayesian Transportation Boundcited by 3
  19. 2019Meta-Learning PAC-Bayes Priors in Model Averagingcited by 1

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 3; 2023: 2; 2024: 2; 2025: 2.

11 papers.

  1. 2021User-friendly introduction to PAC-Bayes boundscited by 234
  2. 2021How Tight Can PAC-Bayes be in the Small Data Regime?cited by 24
  3. 2024More Flexible PAC-Bayesian Meta-Learning by Learning Learning Algorithmscited by 10
  4. 2023Bayes meets Bernstein at the Meta Level: an Analysis of Fast Rates in Meta-Learning with PAC-Bayescited by 8
  5. 2025From Low Intrinsic Dimensionality to Non-Vacuous Generalization Bounds in Deep Multi-Task Learningcited by 5
  6. 2022A General framework for PAC-Bayes Bounds for Meta-Learningcited by 3
  7. 2024Generalization Bounds via Meta-Learned Model Representations: PAC-Bayes and Sample Compression Hypernetworkscited by 3
  8. 2023On the Generalization Error of Meta Learning for the Gibbs Algorithmcited by 2
  9. 2025Fast Rate Bounds for Multi-Task and Meta-Learning with Different Sample Sizescited by 2
  10. 2022Demystify Optimization and Generalization of Over-parameterized PAC-Bayesian Learningcited by 1
  11. 2022Streaming PAC-Bayes Gaussian process regression with a performance guarantee for online decision makingcited by 0