# Expectation Programming: Adapting Probabilistic Programming Systems to Estimate Expectations Efficiently

arXiv: https://arxiv.org/abs/2106.04953 · HTML: https://mlsanity.com/paper/2106.04953/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 54 references that are in the map.
- First posted on arXiv 9 Jun 2021.

## Builds on (most cited first, 13 of 13)

- [Pyro: Deep Universal Probabilistic Programming](https://mlsanity.com/paper/1810.09538/) (2018)
- [A Conceptual Introduction to Hamiltonian Monte Carlo](https://mlsanity.com/paper/1701.02434/) (2017)
- [Rank-normalization, folding, and localization: An improved widehatR for assessing convergence of MCMC](https://mlsanity.com/paper/1903.08008/) (2019)
- [An Introduction to Probabilistic Programming](https://mlsanity.com/paper/1809.10756/) (2018)
- [On the Quantitative Analysis of Decoder-Based Generative Models](https://mlsanity.com/paper/1611.04273/) (2016)
- [Edward: A library for probabilistic modeling, inference, and criticism](https://mlsanity.com/paper/1610.09787/) (2016)
- [A New Approach to Probabilistic Programming Inference](https://mlsanity.com/paper/1507.00996/) (2015)
- [Venture: a higher-order probabilistic programming platform with programmable inference](https://mlsanity.com/paper/1404.0099/) (2014)
- [Semantics for probabilistic programming: higher-order functions, continuous distributions, and soft constraints](https://mlsanity.com/paper/1601.04943/) (2016)
- [Automated learning with a probabilistic programming language: Birch](https://mlsanity.com/paper/1810.01539/) (2018)
- [Measure Transformer Semantics for Bayesian Machine Learning](https://mlsanity.com/paper/1308.0689/) (2013)
- [Composing inference algorithms as program transformations](https://mlsanity.com/paper/1603.01882/) (2016)
- [Amortized Monte Carlo Integration](https://mlsanity.com/paper/1907.08082/) (2019)

## Abstract (arXiv)

We show that the standard computational pipeline of probabilistic programming systems (PPSs) can be inefficient for estimating expectations and introduce the concept of expectation programming to address this. In expectation programming, the aim of the backend inference engine is to directly estimate expected return values of programs, as opposed to approximating their conditional distributions. This distinction, while subtle, allows us to achieve substantial performance improvements over the standard PPS computational pipeline by tailoring computation to the expectation we care about. We realize a particular instance of our expectation programming concept, Expectation Programming in Turing (EPT), by extending the PPS Turing to allow so-called target-aware inference to be run automatically. We then verify the statistical soundness of EPT theoretically, and show that it provides substantial empirical gains in practice.