# Automated Variational Inference in Probabilistic Programming

arXiv: https://arxiv.org/abs/1301.1299 · HTML: https://mlsanity.com/paper/1301.1299/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Jan 2013.

## Built on by (most cited first, 30 of 34)

- [Provable convergence guarantees for black-box variational inference](https://mlsanity.com/paper/2306.03638/) (2023)
- [Mean-field Variational Inference via Wasserstein Gradient Flow](https://mlsanity.com/paper/2207.08074/) (2022)
- [Variational inference with a quantum computer](https://mlsanity.com/paper/2103.06720/) (2021)
- [Nested Variational Inference](https://mlsanity.com/paper/2106.11302/) (2021)
- [CAR-DESPOT: Causally-Informed Online POMDP Planning for Robots in Confounded Environments](https://mlsanity.com/paper/2304.06848/) (2023)
- [Fat-Tailed Variational Inference with Anisotropic Tail Adaptive Flows](https://mlsanity.com/paper/2205.07918/) (2022)
- [Automatic variational inference with cascading flows](https://mlsanity.com/paper/2102.04801/) (2021)
- [Automated Discovery of Pairwise Interactions from Unstructured Data](https://mlsanity.com/paper/2409.07594/) (2024)
- [Learning Proposals for Probabilistic Programs with Inference Combinators](https://mlsanity.com/paper/2103.00668/) (2021)
- [TyXe: Pyro-based Bayesian neural nets for Pytorch](https://mlsanity.com/paper/2110.00276/) (2021)
- [Smoothness Analysis for Probabilistic Programs with Application to Optimised Variational Inference](https://mlsanity.com/paper/2208.10530/) (2022)
- [Sample Average Approximation for Black-Box VI](https://mlsanity.com/paper/2304.06803/) (2023)
- [Deep Quantile Regression for Uncertainty Estimation in Unsupervised and Supervised Lesion Detection](https://mlsanity.com/paper/2109.09374/) (2021)
- [Querying Labelled Data with Scenario Programs for Sim-to-Real Validation](https://mlsanity.com/paper/2112.00206/) (2021)
- [Natural Gradient VI: Guarantees for Non-Conjugate Models](https://mlsanity.com/paper/2510.19163/) (2025)
- [Fast and Correct Gradient-Based Optimisation for Probabilistic Programming via Smoothing](https://mlsanity.com/paper/2301.03415/) (2023)
- [Rethinking Variational Inference for Probabilistic Programs with Stochastic Support](https://mlsanity.com/paper/2311.00594/) (2023)
- [Variational Transdimensional Inference](https://mlsanity.com/paper/2506.04749/) (2025)
- [Variational Bayes survival analysis for unemployment modelling](https://mlsanity.com/paper/2102.02295/) (2021)
- [Variational Elliptical Processes](https://mlsanity.com/paper/2311.12566/) (2023)
- [VISA: Variational Inference with Sequential Sample-Average Approximations](https://mlsanity.com/paper/2403.09429/) (2024)
- [Nearly Dimension-Independent Convergence of Mean-Field Black-Box Variational Inference](https://mlsanity.com/paper/2505.21721/) (2025)
- [Encoding Domain Information with Sparse Priors for Inferring Explainable Latent Variables](https://mlsanity.com/paper/2107.03730/) (2021)
- [Dr. Neurosymbolic, or: How I Learned to Stop Worrying and Accept Statistics](https://mlsanity.com/paper/2209.04049/) (2022)
- [Beyond Bayesian Model Averaging over Paths in Probabilistic Programs with Stochastic Support](https://mlsanity.com/paper/2310.14888/) (2023)
- [Diagonalisation SGD: Fast & Convergent SGD for Non-Differentiable Models via Reparameterisation and Smoothing](https://mlsanity.com/paper/2402.11752/) (2024)
- [COBRA-PPM: A Causal Bayesian Reasoning Architecture Using Probabilistic Programming for Robot Manipulation Under Uncertainty](https://mlsanity.com/paper/2403.14488/) (2024)
- [Massively Parallel Expectation Maximization For Approximate Posteriors](https://mlsanity.com/paper/2503.08264/) (2025)
- [Distributional Sensitivity Analysis: Enabling Differentiability in Sample-Based Inference](https://mlsanity.com/paper/2508.09347/) (2025)
- [Stochastic Gradient Variational Inference with Price's Gradient Estimator from Bures-Wasserstein to Parameter Space](https://mlsanity.com/paper/2602.18718/) (2026)

## Abstract (arXiv)

We present a new algorithm for approximate inference in probabilistic programs, based on a stochastic gradient for variational programs. This method is efficient without restrictions on the probabilistic program; it is particularly practical for distributions which are not analytically tractable, including highly structured distributions that arise in probabilistic programs. We show how to automatically derive mean-field probabilistic programs and optimize them, and demonstrate that our perspective improves inference efficiency over other algorithms.