# Meta-Learning an Inference Algorithm for Probabilistic Programs

arXiv: https://arxiv.org/abs/2103.00737 · HTML: https://mlsanity.com/paper/2103.00737/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 62 references that are in the map.
- First posted on arXiv 1 Mar 2021.

## Builds on (most cited first, 30 of 30)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Pyro: Deep Universal Probabilistic Programming](https://mlsanity.com/paper/1810.09538/) (2018)
- [Expectation Propagation for approximate Bayesian inference](https://mlsanity.com/paper/1301.2294/) (2013)
- [Neural Variational Inference and Learning in Belief Networks](https://mlsanity.com/paper/1402.0030/) (2014)
- [Meta-Learning Probabilistic Inference For Prediction](https://mlsanity.com/paper/1805.09921/) (2018)
- [Learning to Execute](https://mlsanity.com/paper/1410.4615/) (2014)
- [Neural Programmer-Interpreters](https://mlsanity.com/paper/1511.06279/) (2015)
- [Iterative Amortized Inference](https://mlsanity.com/paper/1807.09356/) (2018)
- [Learning Disentangled Representations with Semi-Supervised Deep Generative Models](https://mlsanity.com/paper/1706.00400/) (2017)
- [An Introduction to Probabilistic Programming](https://mlsanity.com/paper/1809.10756/) (2018)
- [Neural Execution Engines: Learning to Execute Subroutines](https://mlsanity.com/paper/2006.08084/) (2020)
- [Learning to Execute Programs with Instruction Pointer Attention Graph Neural Networks](https://mlsanity.com/paper/2010.12621/) (2020)
- [Inference Compilation and Universal Probabilistic Programming](https://mlsanity.com/paper/1610.09900/) (2016)
- [Edward: A library for probabilistic modeling, inference, and criticism](https://mlsanity.com/paper/1610.09787/) (2016)
- [Deep Amortized Inference for Probabilistic Programs](https://mlsanity.com/paper/1610.05735/) (2016)
- [A New Approach to Probabilistic Programming Inference](https://mlsanity.com/paper/1507.00996/) (2015)
- [Meta-Amortized Variational Inference and Learning](https://mlsanity.com/paper/1902.01950/) (2019)
- [Venture: a higher-order probabilistic programming platform with programmable inference](https://mlsanity.com/paper/1404.0099/) (2014)
- [Inference Networks for Sequential Monte Carlo in Graphical Models](https://mlsanity.com/paper/1602.06701/) (2016)
- [Layered Adaptive Importance Sampling](https://mlsanity.com/paper/1505.04732/) (2015)
- [Simple, Distributed, and Accelerated Probabilistic Programming](https://mlsanity.com/paper/1811.02091/) (2018)
- [Meta-Learning for Stochastic Gradient MCMC](https://mlsanity.com/paper/1806.04522/) (2018)
- [Design and Implementation of Probabilistic Programming Language Anglican](https://mlsanity.com/paper/1608.05263/) (2016)
- [Meta-Learning with Shared Amortized Variational Inference](https://mlsanity.com/paper/2008.12037/) (2020)
- [Adaptive Gaussian process approximation for Bayesian inference with expensive likelihood functions](https://mlsanity.com/paper/1703.09930/) (2017)
- [Automatic variational inference with cascading flows](https://mlsanity.com/paper/2102.04801/) (2021)
- [Efficient Probabilistic Inference in the Quest for Physics Beyond the Standard Model](https://mlsanity.com/paper/1807.07706/) (2018)
- [Kernel-Based Just-In-Time Learning for Passing Expectation Propagation Messages](https://mlsanity.com/paper/1503.02551/) (2015)
- [Probabilistic programs for inferring the goals of autonomous agents](https://mlsanity.com/paper/1704.04977/) (2017)
- [Time Series Structure Discovery via Probabilistic Program Synthesis](https://mlsanity.com/paper/1611.07051/) (2016)

## Built on by (most cited first, 1 of 1)

- [Foundation Posteriors for Approximate Probabilistic Inference](https://mlsanity.com/paper/2205.09735/) (2022)

## Abstract (arXiv)

We present a meta-algorithm for learning a posterior-inference algorithm for restricted probabilistic programs. Our meta-algorithm takes a training set of probabilistic programs that describe models with observations, and attempts to learn an efficient method for inferring the posterior of a similar program. A key feature of our approach is the use of what we call a white-box inference algorithm that extracts information directly from model descriptions themselves, given as programs. Concretely, our white-box inference algorithm is equipped with multiple neural networks, one for each type of atomic command, and computes an approximate posterior of a given probabilistic program by analysing individual atomic commands in the program using these networks. The parameters of the networks are learnt from a training set by our meta-algorithm. We empirically demonstrate that the learnt inference algorithm generalises well to programs that are new in terms of both parameters and model structures, and report cases where our approach achieves greater test-time efficiency than alternative approaches such as HMC. The overall results show the promise as well as remaining challenges of our approach.