# Recasting Gradient-Based Meta-Learning as Hierarchical Bayes

arXiv: https://arxiv.org/abs/1801.08930 · HTML: https://mlsanity.com/paper/1801.08930/

## Key facts

- Cited by 180 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Jan 2018.

## Built on by (most cited first, 30 of 180)

- [Efficiently Identifying Task Groupings for Multi-Task Learning](https://mlsanity.com/paper/2109.04617/) (2021)
- [A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities](https://mlsanity.com/paper/2205.06743/) (2022)
- [Priors in Bayesian Deep Learning: A Review](https://mlsanity.com/paper/2105.06868/) (2021)
- [Self-Support Few-Shot Semantic Segmentation](https://mlsanity.com/paper/2207.11549/) (2022)
- [Building Machines that Learn and Think with People](https://mlsanity.com/paper/2408.03943/) (2024)
- [Robot Learning from Randomized Simulations: A Review](https://mlsanity.com/paper/2111.00956/) (2021)
- [Continual Learning via Local Module Composition](https://mlsanity.com/paper/2111.07736/) (2021)
- [Sharp-MAML: Sharpness-Aware Model-Agnostic Meta Learning](https://mlsanity.com/paper/2206.03996/) (2022)
- [Bridging Multi-Task Learning and Meta-Learning: Towards Efficient Training and Effective Adaptation](https://mlsanity.com/paper/2106.09017/) (2021)
- [Meta-Learning with Task-Adaptive Loss Function for Few-Shot Learning](https://mlsanity.com/paper/2110.03909/) (2021)
- [Shallow Bayesian Meta Learning for Real-World Few-Shot Recognition](https://mlsanity.com/paper/2101.02833/) (2021)
- [Object Representations as Fixed Points: Training Iterative Refinement Algorithms with Implicit Differentiation](https://mlsanity.com/paper/2207.00787/) (2022)
- [Meta-Learning with Fewer Tasks through Task Interpolation](https://mlsanity.com/paper/2106.02695/) (2021)
- [Bootstrapped Meta-Learning](https://mlsanity.com/paper/2109.04504/) (2021)
- [Generalization Bounds For Meta-Learning: An Information-Theoretic Analysis](https://mlsanity.com/paper/2109.14595/) (2021)
- [Modeling rapid language learning by distilling Bayesian priors into artificial neural networks](https://mlsanity.com/paper/2305.14701/) (2023)
- [Uncertainty-guided Model Generalization to Unseen Domains](https://mlsanity.com/paper/2103.07531/) (2021)
- [Meta-Learning Dynamics Forecasting Using Task Inference](https://mlsanity.com/paper/2102.10271/) (2021)
- [Few-Shot Diffusion Models](https://mlsanity.com/paper/2205.15463/) (2022)
- [Learning a Prior over Intent via Meta-Inverse Reinforcement Learning](https://mlsanity.com/paper/1805.12573/) (2018)
- [Alchemy: A benchmark and analysis toolkit for meta-reinforcement learning agents](https://mlsanity.com/paper/2102.02926/) (2021)
- [Generalization of Model-Agnostic Meta-Learning Algorithms: Recurring and Unseen Tasks](https://mlsanity.com/paper/2102.03832/) (2021)
- [Learning Domain Invariant Prompt for Vision-Language Models](https://mlsanity.com/paper/2212.04196/) (2022)
- [FedPop: A Bayesian Approach for Personalised Federated Learning](https://mlsanity.com/paper/2206.03611/) (2022)
- [Gradient Descent on Neurons and its Link to Approximate Second-Order Optimization](https://mlsanity.com/paper/2201.12250/) (2022)
- [A Closer Look at Learned Optimization: Stability, Robustness, and Inductive Biases](https://mlsanity.com/paper/2209.11208/) (2022)
- [From partners to populations: A hierarchical Bayesian account of coordination and convention](https://mlsanity.com/paper/2104.05857/) (2021)
- [Meta-Learning to Improve Pre-Training](https://mlsanity.com/paper/2111.01754/) (2021)
- [Recurrence without Recurrence: Stable Video Landmark Detection with Deep Equilibrium Models](https://mlsanity.com/paper/2304.00600/) (2023)
- [Prompt Learning via Meta-Regularization](https://mlsanity.com/paper/2404.00851/) (2024)

## Abstract (arXiv)

Meta-learning allows an intelligent agent to leverage prior learning episodes as a basis for quickly improving performance on a novel task. Bayesian hierarchical modeling provides a theoretical framework for formalizing meta-learning as inference for a set of parameters that are shared across tasks. Here, we reformulate the model-agnostic meta-learning algorithm (MAML) of Finn et al. (2017) as a method for probabilistic inference in a hierarchical Bayesian model. In contrast to prior methods for meta-learning via hierarchical Bayes, MAML is naturally applicable to complex function approximators through its use of a scalable gradient descent procedure for posterior inference. Furthermore, the identification of MAML as hierarchical Bayes provides a way to understand the algorithm's operation as a meta-learning procedure, as well as an opportunity to make use of computational strategies for efficient inference. We use this opportunity to propose an improvement to the MAML algorithm that makes use of techniques from approximate inference and curvature estimation.