mlsanityData to 7 Oct 2026

Generalization Bounds For Meta-Learning: An Information-Theoretic Analysis

arXiv 2109.14595 ↗First posted 29 Sep 2021cs.LG stat.MLcited by 45

Not on the map. Why?

Abstract (arXiv)

We derive a novel information-theoretic analysis of the generalization property of meta-learning algorithms. Concretely, our analysis proposes a generic understanding of both the conventional learning-to-learn framework and the modern model-agnostic meta-learning (MAML) algorithms. Moreover, we provide a data-dependent generalization bound for a stochastic variant of MAML, which is non-vacuous for deep few-shot learning. As compared to previous bounds that depend on the square norm of gradients, empirical validations on both simulated data and a well-known few-shot benchmark show that our bound is orders of magnitude tighter in most situations.

Key facts. Cited by 45 papers in the map (data through 7 Oct 2026). Builds on 35 of its 52 references that are in the map. First posted on arXiv 29 Sep 2021.

Builds on

35 of 52 references.

References · 35

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  3. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  4. 2016Matching Networks for One Shot Learningcited by 2,246
  5. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  6. 2017Learning to Compare: Relation Network for Few-Shot Learningcited by 1,139
  7. 2018Lipschitz regularity of deep neural networks: analysis and efficient estimationcited by 413
  8. 2017Few-Shot Learning with Graph Neural Networkscited by 324
  9. 2020Personalized Federated Learning: A Meta-Learning Approachcited by 309
  10. 2017Information-theoretic analysis of generalization capability of learning algorithmscited by 299
  11. 2019Improving Federated Learning Personalization via Model Agnostic Meta Learningcited by 264
  12. 2018Probabilistic Model-Agnostic Meta-Learningcited by 254
23 more
  1. 2017Non-convex learning via Stochastic Gradient Langevin Dynamics: a nonasymptotic analysiscited by 242
  2. 2015The Benefit of Multitask Representation Learningcited by 194
  3. 2018Bayesian Model-Agnostic Meta-Learningcited by 194
  4. 2018Formal Limitations on the Measurement of Mutual Informationcited by 189
  5. 2019Adaptive Gradient-Based Meta-Learning Methodscited by 183
  6. 2018Recasting Gradient-Based Meta-Learning as Hierarchical Bayescited by 180
  7. 2020Reasoning About Generalization via Conditional Mutual Informationcited by 145
  8. 2019Online Meta-Learningcited by 137
  9. 2019On the Convergence Theory of Gradient-Based Model-Agnostic Meta-Learning Algorithmscited by 132
  10. 2019Information-Theoretic Generalization Bounds for SGLD via Data-Dependent Estimatescited by 118
  11. 2019Tightening Mutual Information Based Bounds on Generalization Errorcited by 117
  12. 2015How much does your data exploration overfit? Controlling bias via information usagecited by 95
  13. 2016PAC-Bayesian Theory Meets Bayesian Inferencecited by 87
  14. 2018Generalization Error Bounds for Noisy, Iterative Algorithmscited by 79
  15. 2017Meta-Learning by Adjusting Priors Based on Extended PAC-Bayes Theorycited by 76
  16. 2013A PAC-Bayesian bound for Lifelong Learningcited by 69
  17. 2019Provable Guarantees for Gradient-Based Meta-Learningcited by 67
  18. 2019On Generalization Error Bounds of Noisy Gradient Methods for Non-Convex Learningcited by 57
  19. 2019Learning-to-Learn Stochastic Gradient Descent with Biased Regularizationcited by 49
  20. 2020Information-Theoretic Generalization Bounds for Meta-Learning and Applicationscited by 32
  21. 2020How Important is the Train-Validation Split in Meta-Learning?cited by 30
  22. 2020Theoretical Convergence of Multi-Step Model-Agnostic Meta-Learningcited by 28
  23. 2021A Representation Learning Perspective on the Importance of Train-Validation Splitting in Meta-Learningcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 10; 2023: 11; 2024: 7; 2025: 13; 2026: 4.

45 papers.

Sort:

By area:

  1. 2022On Learning Fairness and Accuracy on Multiple Subgroupscited by 31
  2. 2023Out-of-Domain Robustness via Targeted Augmentationscited by 24
  3. 2022Information-Theoretic Analysis of Unsupervised Domain Adaptationcited by 15
  4. 2022Is Bayesian Model-Agnostic Meta Learning Better than Model-Agnostic Meta Learning, Provably?cited by 13
  5. 2022Understanding Benign Overfitting in Gradient-Based Meta Learningcited by 11
  6. 2022Evaluated CMI Bounds for Meta Learning: Tightness and Expressivenesscited by 11
  7. 2022Scalable PAC-Bayesian Meta-Learning via the PAC-Optimal Hyper-Posterior: From Theory to Practicecited by 11
  8. 2023Metalearning with Very Few Samples Per Taskcited by 11
  9. 2022The Curse of Low Task Diversity: On the Failure of Transfer Learning to Outperform MAML and Their Empirical Equivalencecited by 10
  10. 2023Architecture, Dataset and Model-Scale Agnostic Data-free Meta-Learningcited by 10
  11. 2024More Flexible PAC-Bayesian Meta-Learning by Learning Learning Algorithmscited by 10
  12. 2022Provable Generalization of Overparameterized Meta-learning Trained with SGDcited by 9
33 more
  1. 2022Information bottleneck theory of high-dimensional regression: relevancy, efficiency and optimalitycited by 9
  2. 2023A Simple Yet Effective Strategy to Robustify the Meta Learning Paradigmcited by 9
  3. 2023Bayes meets Bernstein at the Meta Level: an Analysis of Fast Rates in Meta-Learning with PAC-Bayescited by 8
  4. 2023Algorithm-Dependent Bounds for Representation Learning of Multi-Source Domain Adaptationcited by 7
  5. 2023Meta-learning Control Variates: Variance Reduction with Limited Datacited by 6
  6. 2023Is Pre-training Truly Better Than Meta-Learning?cited by 6
  7. 2025Meta-Learning for Physically-Constrained Neural System Identificationcited by 5
  8. 2025Learning to Learn Weight Generation via Local Consistency Diffusioncited by 4
  9. 2025Is Meta-Learning Out? Rethinking Unsupervised Few-Shot Classification with Limited Entropycited by 4
  10. 2024How Does Distribution Matching Help Domain Generalization: An Information-theoretic Analysiscited by 3
  11. 2024Pre-Training and Personalized Fine-Tuning via Over-the-Air Federated Meta-Learning: Convergence-Generalization Trade-Offscited by 3
  12. 2024Theoretical Investigations and Practical Enhancements on Tail Task Risk Minimization in Meta Learningcited by 3
  13. 2025Fairness Overfitting in Machine Learning: An Information-Theoretic Perspectivecited by 3
  14. 2025On The Statistical Limits of Self-Improving Agentscited by 3
  15. 2022A Statistical Model for Predicting Generalization in Few-Shot Classificationcited by 2
  16. 2023On the Generalization Error of Meta Learning for the Gibbs Algorithmcited by 2
  17. 2023Information-Theoretic Generalization Bounds for Transductive Learning and its Applicationscited by 2
  18. 2025The Capabilities and Limitations of Weak-to-Strong Generalization: Generalization and Calibrationcited by 2
  19. 2025Generalization in VAE and Diffusion Models: A Unified Information-Theoretic Analysiscited by 2
  20. 2024On the ERM Principle in Meta-Learningcited by 1
  21. 2025Transformers are almost optimal metalearners for linear classificationcited by 1
  22. 2023Improving Generalization in Meta-Learning via Meta-Gradient Augmentationcited by 0
  23. 2024Transfer learning of state-based potential games for process optimization in decentralized manufacturing systemscited by 0
  24. 2024Neuromodulated Meta-Learningcited by 0
  25. 2025Efficient Generalization via Multimodal Co-Training under Data Scarcity and Distribution Shiftcited by 0
  26. 2025Information Theory in Open-world Machine Learning Foundations, Frameworks, and Future Directioncited by 0
  27. 2025An Information-Theoretic Analysis of OOD Generalization in Meta-Reinforcement Learningcited by 0
  28. 2025Representative Action Selection for Large Action Space: From Bandits to MDPscited by 0
  29. 2025Informing Acquisition Functions via Foundation Models for Molecular Discoverycited by 0
  30. 2026Bayesian Meta-Learning with Expert Feedback for Task-Shift Adaptation through Causal Embeddingscited by 0
  31. 2026On the Stability and Generalization of First-order Bilevel Minimax Optimizationcited by 0
  32. 2026Gradient Transformer: Learning to Generate Updates for LLMscited by 0
  33. 2026Provable Data Scaling Law for Meta Learning via Complexity Minimizationcited by 0