# How To Train Your Program: a Probabilistic Programming Pattern for Bayesian Learning From Data

arXiv: https://arxiv.org/abs/2105.03650 · HTML: https://mlsanity.com/paper/2105.03650/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 34 references that are in the map.
- First posted on arXiv 8 May 2021.

## Builds on (most cited first, 13 of 13)

- [Stochastic Gradient Hamiltonian Monte Carlo](https://mlsanity.com/paper/1402.4102/) (2014)
- [Coresets for Scalable Bayesian Logistic Regression](https://mlsanity.com/paper/1605.06423/) (2016)
- [On Markov chain Monte Carlo methods for tall data](https://mlsanity.com/paper/1505.02827/) (2015)
- [Inference Compilation and Universal Probabilistic Programming](https://mlsanity.com/paper/1610.09900/) (2016)
- [Sparse Variational Inference: Bayesian Coresets from Scratch](https://mlsanity.com/paper/1906.03329/) (2019)
- [Austerity in MCMC Land: Cutting the Metropolis-Hastings Budget](https://mlsanity.com/paper/1304.5299/) (2013)
- [Firefly Monte Carlo: Exact MCMC with Subsets of Data](https://mlsanity.com/paper/1403.5693/) (2014)
- [Etalumis: Bringing Probabilistic Programming to Scientific Simulators at Scale](https://mlsanity.com/paper/1907.03382/) (2019)
- [Probabilistic Programs with Stochastic Conditioning](https://mlsanity.com/paper/2010.00282/) (2020)
- [Subsampling MCMC - An introduction for the survey statistician](https://mlsanity.com/paper/1807.08409/) (2018)
- [Deployable probabilistic programming](https://mlsanity.com/paper/1906.11199/) (2019)
- [Learning Probabilistic Programs](https://mlsanity.com/paper/1407.2646/) (2014)
- [Inference Over Programs That Make Predictions](https://mlsanity.com/paper/1810.01190/) (2018)

## Abstract (arXiv)

We present a Bayesian approach to machine learning with probabilistic programs. In our approach, training on available data is implemented as inference on a hierarchical model. The posterior distribution of model parameters is then used to textitstochastically condition a complementary model, such that inference on new data yields the same posterior distribution of latent parameters corresponding to the new data as inference on a hierachical model on the combination of both previously available and new data, at a lower computation cost. We frame the approach as a design pattern of probabilistic programming referred to herein as `stump and fungus', and evaluate realization of the pattern on case studies.