# Deep Amortized Inference for Probabilistic Programs

arXiv: https://arxiv.org/abs/1610.05735 · HTML: https://mlsanity.com/paper/1610.05735/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Oct 2016.

## Built on by (most cited first, 23 of 23)

- [Transformers Can Do Bayesian Inference](https://mlsanity.com/paper/2112.10510/) (2021)
- [Combining Induction and Transduction for Abstract Reasoning](https://mlsanity.com/paper/2411.02272/) (2024)
- [Variational Deep Image Restoration](https://mlsanity.com/paper/2207.01074/) (2022)
- [AutoToM: Scaling Model-based Mental Inference via Automated Agent Modeling](https://mlsanity.com/paper/2502.15676/) (2025)
- [A Structural Causal Model for MR Images of Multiple Sclerosis](https://mlsanity.com/paper/2103.03158/) (2021)
- [Amortized Projection Optimization for Sliced Wasserstein Generative Models](https://mlsanity.com/paper/2203.13417/) (2022)
- [Memory-Based Meta-Learning on Non-Stationary Distributions](https://mlsanity.com/paper/2302.03067/) (2023)
- [Nested Variational Inference](https://mlsanity.com/paper/2106.11302/) (2021)
- [Amortized Bayesian Multilevel Models](https://mlsanity.com/paper/2408.13230/) (2024)
- [Graphically Structured Diffusion Models](https://mlsanity.com/paper/2210.11633/) (2022)
- [Does Unsupervised Domain Adaptation Improve the Robustness of Amortized Bayesian Inference? A Systematic Evaluation](https://mlsanity.com/paper/2502.04949/) (2025)
- [Learning Proposals for Probabilistic Programs with Inference Combinators](https://mlsanity.com/paper/2103.00668/) (2021)
- [Smoothness Analysis for Probabilistic Programs with Application to Optimised Variational Inference](https://mlsanity.com/paper/2208.10530/) (2022)
- [Estimators of Entropy and Information via Inference in Probabilistic Models](https://mlsanity.com/paper/2202.12363/) (2022)
- [Amortized Bayesian Mixture Models](https://mlsanity.com/paper/2501.10229/) (2025)
- [From Mice to Trains: Amortized Bayesian Inference on Graph Data](https://mlsanity.com/paper/2601.02241/) (2026)
- [Meta-Learning an Inference Algorithm for Probabilistic Programs](https://mlsanity.com/paper/2103.00737/) (2021)
- [flip-hoisting: Exploiting Repeated Parameters in Discrete Probabilistic Programs](https://mlsanity.com/paper/2110.10284/) (2021)
- [The Urysohn Machine: A Metric-Topological Model of Computation](https://mlsanity.com/paper/2508.14143/) (2025)
- [Variational Deep Image Denoising](https://mlsanity.com/paper/2104.00965/) (2021)
- [Inference Plans for Hybrid Particle Filtering](https://mlsanity.com/paper/2408.11283/) (2024)
- [AutoBayes: A Compositional Framework for Generalized Variational Inference](https://mlsanity.com/paper/2503.18608/) (2025)
- [Amortised adaptive importance sampling](https://mlsanity.com/paper/2610.16736/) (2026)

## Abstract (arXiv)

Probabilistic programming languages (PPLs) are a powerful modeling tool, able to represent any computable probability distribution. Unfortunately, probabilistic program inference is often intractable, and existing PPLs mostly rely on expensive, approximate sampling-based methods. To alleviate this problem, one could try to learn from past inferences, so that future inferences run faster. This strategy is known as amortized inference; it has recently been applied to Bayesian networks and deep generative models. This paper proposes a system for amortized inference in PPLs. In our system, amortization comes in the form of a parameterized guide program. Guide programs have similar structure to the original program, but can have richer data flow, including neural network components. These networks can be optimized so that the guide approximately samples from the posterior distribution defined by the original program. We present a flexible interface for defining guide programs and a stochastic gradient-based scheme for optimizing guide parameters, as well as some preliminary results on automatically deriving guide programs. We explore in detail the common machine learning pattern in which a 'local' model is specified by 'global' random values and used to generate independent observed data points; this gives rise to amortized local inference supporting global model learning.