mlsanityData to 14 Oct 2026

Provable Guarantees for Gradient-Based Meta-Learning

arXiv 1902.10644 ↗First posted 27 Feb 2019cs.LG cs.AI stat.MLcited by 67

Not on the map. Why?

Abstract (arXiv)

We study the problem of meta-learning through the lens of online convex optimization, developing a meta-algorithm bridging the gap between popular gradient-based meta-learning and classical regularization-based multi-task transfer methods. Our method is the first to simultaneously satisfy good sample efficiency guarantees in the convex setting, with generalization bounds that improve with task-similarity, while also being computationally scalable to modern deep learning architectures and the many-task setting. Despite its simplicity, the algorithm matches, up to a constant factor, a lower bound on the performance of any such parameter-transfer method under natural task similarity assumptions. We use experiments in both convex and deep learning settings to verify and demonstrate the applicability of our theory.

Key facts. Cited by 67 papers in the map (data through 14 Oct 2026). First posted on arXiv 27 Feb 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 25; 2022: 19; 2023: 10; 2024: 7; 2025: 3; 2026: 3.

67 papers.

Sort:

By area:

  1. 2021Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyondcited by 153
  2. 2021What Do We Mean by Generalization in Federated Learning?cited by 62
  3. 2021Federated Hyperparameter Tuning: Challenges, Baselines, and Connections to Weight-Sharingcited by 58
  4. 2021Convergence and Accuracy Trade-Offs in Federated Learning and Meta-Learningcited by 52
  5. 2021Bridging Multi-Task Learning and Meta-Learning: Towards Efficient Training and Effective Adaptationcited by 50
  6. 2021Bootstrapped Meta-Learningcited by 45
  7. 2021Generalization Bounds For Meta-Learning: An Information-Theoretic Analysiscited by 45
  8. 2023An Introduction to Bi-level Optimization: Foundations and Applications in Signal Processing and Machine Learningcited by 41
  9. 2022Adaptive and Robust Multi-Task Learningcited by 34
  10. 2021Generalization Bounds for Meta-Learning via PAC-Bayes and Uniform Stabilitycited by 30
  11. 2022Global Convergence of MAML and Theory-Inspired Neural Architecture Search for Few-Shot Learningcited by 22
  12. 2021On Data Efficiency of Meta-learningcited by 16
38 more
  1. 2024Imperative Learning: A Self-supervised Neuro-Symbolic Learning Framework for Robot Autonomycited by 16
  2. 2021Meta-Learning with Neural Tangent Kernelscited by 15
  3. 2021Learning-to-learn non-convex piecewise-Lipschitz functionscited by 15
  4. 2021Provable Lifelong Learning of Representationscited by 15
  5. 2022Provable and Efficient Continual Representation Learningcited by 14
  6. 2022Non-Stationary Representation Learning in Sequential Linear Banditscited by 13
  7. 2022Is Bayesian Model-Agnostic Meta Learning Better than Model-Agnostic Meta Learning, Provably?cited by 13
  8. 2023Towards Constituting Mathematical Structures for Learning to Optimizecited by 13
  9. 2021Conditional Meta-Learning of Linear Representationscited by 12
  10. 2021Compositional federated learning: Applications in distributionally robust averaging and meta learningcited by 12
  11. 2021A Representation Learning Perspective on the Importance of Train-Validation Splitting in Meta-Learningcited by 12
  12. 2024A CMDP-within-online framework for Meta-Safe Reinforcement Learningcited by 12
  13. 2022Understanding Benign Overfitting in Gradient-Based Meta Learningcited by 11
  14. 2022Evaluated CMI Bounds for Meta Learning: Tightness and Expressivenesscited by 11
  15. 2023Metalearning with Very Few Samples Per Taskcited by 11
  16. 2021Learning an Explicit Hyperparameter Prediction Function Conditioned on Taskscited by 9
  17. 2021A Regret Minimization Approach to Iterative Learning Controlcited by 8
  18. 2021Online Continual Adaptation with Active Self-Trainingcited by 8
  19. 2021Meta Learning on a Sequence of Imbalanced Domains with Difficulty Awarenesscited by 8
  20. 2022Non-stationary Bandits and Meta-Learning with a Small Set of Optimal Armscited by 8
  21. 2023Convergence of First-Order Algorithms for Meta-Learning with Moreau Envelopescited by 8
  22. 2023Bayes meets Bernstein at the Meta Level: an Analysis of Fast Rates in Meta-Learning with PAC-Bayescited by 8
  23. 2021Meta-Learning with Graph Neural Networks: Methods and Applicationscited by 7
  24. 2022On Enforcing Better Conditioned Meta-Learning for Rapid Few-Shot Adaptationcited by 7
  25. 2023Best-Effort Adaptationcited by 7
  26. 2024In-Trajectory Inverse Reinforcement Learning: Learn Incrementally Before An Ongoing Trajectory Terminatescited by 7
  27. 2021Meta-strategy for Learning Tuning Parameters with Guaranteescited by 6
  28. 2021Multitask Online Mirror Descentcited by 6
  29. 2021Memory-Based Optimization Methods for Model-Agnostic Meta-Learning and Personalized Federated Learningcited by 6
  30. 2022Meta-Learning Adversarial Banditscited by 5
  31. 2022Meta-Learning Online Control for Linear Dynamical Systemscited by 5
  32. 2022Generalization Properties of Retrieval-based Modelscited by 5
  33. 2024Data-Driven Performance Guarantees for Classical and Learned Optimizerscited by 5
  34. 2022Meta Learning MDPs with Linear Transition Modelscited by 4
  35. 2023Hypothesis Transfer Learning with Surrogate Classification Losses: Generalization Bounds through Algorithmic Stabilitycited by 4
  36. 2024Help Me Help You: The Aggregate Value of Source and Target Data in Transfer Learningcited by 4
  37. 2021Debiasing a First-order Heuristic for Approximate Bi-level Optimizationcited by 3
  38. 2022A General framework for PAC-Bayes Bounds for Meta-Learningcited by 3

50 most cited of 67 All 67 (JSON)