# Inference Compilation and Universal Probabilistic Programming

arXiv: https://arxiv.org/abs/1610.09900 · HTML: https://mlsanity.com/paper/1610.09900/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Oct 2016.

## Built on by (most cited first, 30 of 32)

- [Benchmarking Simulation-Based Inference](https://mlsanity.com/paper/2101.04653/) (2021)
- [All-in-one simulation-based inference](https://mlsanity.com/paper/2404.09636/) (2024)
- [Combining Induction and Transduction for Abstract Reasoning](https://mlsanity.com/paper/2411.02272/) (2024)
- [Human-like Few-Shot Learning via Bayesian Reasoning over Natural Language](https://mlsanity.com/paper/2306.02797/) (2023)
- [Amortized Bayesian Multilevel Models](https://mlsanity.com/paper/2408.13230/) (2024)
- [ProbNeRF: Uncertainty-Aware Inference of 3D Shapes from 2D Images](https://mlsanity.com/paper/2210.17415/) (2022)
- [Graphically Structured Diffusion Models](https://mlsanity.com/paper/2210.11633/) (2022)
- [Robust Amortized Bayesian Inference with Self-Consistency Losses on Unlabeled Data](https://mlsanity.com/paper/2501.13483/) (2025)
- [Does Unsupervised Domain Adaptation Improve the Robustness of Amortized Bayesian Inference? A Systematic Evaluation](https://mlsanity.com/paper/2502.04949/) (2025)
- [Compositional simulation-based inference for time series](https://mlsanity.com/paper/2411.02728/) (2024)
- [BayesFlow 2: Multi-Backend Amortized Bayesian Inference in Python](https://mlsanity.com/paper/2602.07098/) (2026)
- [Recursive Monte Carlo and Variational Inference with Auxiliary Variables](https://mlsanity.com/paper/2203.02836/) (2022)
- [Neural Amortized Inference for Nested Multi-agent Reasoning](https://mlsanity.com/paper/2308.11071/) (2023)
- [Foundation Posteriors for Approximate Probabilistic Inference](https://mlsanity.com/paper/2205.09735/) (2022)
- [Uncertain Evidence in Probabilistic Models and Stochastic Simulators](https://mlsanity.com/paper/2210.12236/) (2022)
- [Comparison of Affine and Rational Quadratic Spline Coupling and Autoregressive Flows through Robust Statistical Tests](https://mlsanity.com/paper/2302.12024/) (2023)
- [Machine Learning and the Future of Bayesian Computation](https://mlsanity.com/paper/2304.11251/) (2023)
- [Estimators of Entropy and Information via Inference in Probabilistic Models](https://mlsanity.com/paper/2202.12363/) (2022)
- [Bayesian Program Learning by Decompiling Amortized Knowledge](https://mlsanity.com/paper/2306.07856/) (2023)
- [Rethinking Variational Inference for Probabilistic Programs with Stochastic Support](https://mlsanity.com/paper/2311.00594/) (2023)
- [Meta-Learning an Inference Algorithm for Probabilistic Programs](https://mlsanity.com/paper/2103.00737/) (2021)
- [A Unifying Framework for Unsupervised Concept Extraction](https://mlsanity.com/paper/2604.24936/) (2026)
- [Control-Data Separation and Logical Condition Propagation for Efficient Inference on Probabilistic Programs](https://mlsanity.com/paper/2101.01502/) (2021)
- [How To Train Your Program: a Probabilistic Programming Pattern for Bayesian Learning From Data](https://mlsanity.com/paper/2105.03650/) (2021)
- [Unifying AI Algorithms with Probabilistic Programming using Implicitly Defined Representations](https://mlsanity.com/paper/2110.02325/) (2021)
- [Push: Concurrent Probabilistic Programming for Bayesian Deep Learning](https://mlsanity.com/paper/2306.06528/) (2023)
- [Beyond Bayesian Model Averaging over Paths in Probabilistic Programs with Stochastic Support](https://mlsanity.com/paper/2310.14888/) (2023)
- [Efficient Incremental Belief Updates Using Weighted Virtual Observations](https://mlsanity.com/paper/2402.06940/) (2024)
- [Amortized Factor Inference Networks for Posterior Inference](https://mlsanity.com/paper/2605.26419/) (2026)
- [Scalable Amortized Variational Inference for Non-Poisson Buy-'Til-You-Die Models](https://mlsanity.com/paper/2608.19022/) (2026)

## Abstract (arXiv)

We introduce a method for using deep neural networks to amortize the cost of inference in models from the family induced by universal probabilistic programming languages, establishing a framework that combines the strengths of probabilistic programming and deep learning methods. We call what we do "compilation of inference" because our method transforms a denotational specification of an inference problem in the form of a probabilistic program written in a universal programming language into a trained neural network denoted in a neural network specification language. When at test time this neural network is fed observational data and executed, it performs approximate inference in the original model specified by the probabilistic program. Our training objective and learning procedure are designed to allow the trained neural network to be used as a proposal distribution in a sequential importance sampling inference engine. We illustrate our method on mixture models and Captcha solving and show significant speedups in the efficiency of inference.