# Meta-Learning with Variational Bayes

arXiv: https://arxiv.org/abs/2103.02265 · HTML: https://mlsanity.com/paper/2103.02265/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 50 references that are in the map.
- First posted on arXiv 3 Mar 2021.

## Builds on (most cited first, 23 of 23)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [Optimizing Neural Networks with Kronecker-factored Approximate Curvature](https://mlsanity.com/paper/1503.05671/) (2015)
- [TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems](https://mlsanity.com/paper/1603.04467/) (2016)
- [Meta-Learning with Latent Embedding Optimization](https://mlsanity.com/paper/1807.05960/) (2018)
- [Neural Processes](https://mlsanity.com/paper/1807.01622/) (2018)
- [Continual Unsupervised Representation Learning](https://mlsanity.com/paper/1910.14481/) (2019)
- [Towards a Neural Statistician](https://mlsanity.com/paper/1606.02185/) (2016)
- [Composing graphical models with neural networks for structured representations and fast inference](https://mlsanity.com/paper/1603.06277/) (2016)
- [Implicit Reparameterization Gradients](https://mlsanity.com/paper/1805.08498/) (2018)
- [Deep Online Learning via Meta-Learning: Continual Adaptation for Model-Based RL](https://mlsanity.com/paper/1812.07671/) (2018)
- [Hindsight Credit Assignment](https://mlsanity.com/paper/1912.02503/) (2019)
- [Unsupervised Predictive Memory in a Goal-Directed Agent](https://mlsanity.com/paper/1803.10760/) (2018)
- [Memory-based Parameter Adaptation](https://mlsanity.com/paper/1802.10542/) (2018)
- [Variational Message Passing with Structured Inference Networks](https://mlsanity.com/paper/1803.05589/) (2018)
- [The Kanerva Machine: A Generative Distributed Memory](https://mlsanity.com/paper/1804.01756/) (2018)
- [Learning Attractor Dynamics for Generative Memory](https://mlsanity.com/paper/1811.09556/) (2018)
- [Product Kanerva Machines: Factorized Bayesian Memory](https://mlsanity.com/paper/2002.02385/) (2020)
- [VQ-DRAW: A Sequential Discrete VAE](https://mlsanity.com/paper/2003.01599/) (2020)

## Abstract (arXiv)

The field of meta-learning seeks to improve the ability of today's machine learning systems to adapt efficiently to small amounts of data. Typically this is accomplished by training a system with a parametrized update rule to improve a task-relevant objective based on supervision or a reward function. However, in many domains of practical interest, task data is unlabeled, or reward functions are unavailable. In this paper we introduce a new approach to address the more general problem of generative meta-learning, which we argue is an important prerequisite for obtaining human-level cognitive flexibility in artificial agents, and can benefit many practical applications along the way. Our contribution leverages the AEVB framework and mean-field variational Bayes, and creates fast-adapting latent-space generative models. At the heart of our contribution is a new result, showing that for a broad class of deep generative latent variable models, the relevant VB updates do not depend on any generative neural network. The theoretical merits of our approach are reflected in empirical experiments.