# Venture: a higher-order probabilistic programming platform with programmable inference

arXiv: https://arxiv.org/abs/1404.0099 · HTML: https://mlsanity.com/paper/1404.0099/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Apr 2014.

## Built on by (most cited first, 20 of 20)

- [From Word Models to World Models: Translating from Natural Language to the Probabilistic Language of Thought](https://mlsanity.com/paper/2306.12672/) (2023)
- [Self-Steering Language Models](https://mlsanity.com/paper/2504.07081/) (2025)
- [DomiKnowS: A Library for Integration of Symbolic Domain Knowledge in Deep Learning](https://mlsanity.com/paper/2108.12370/) (2021)
- [Recursive Monte Carlo and Variational Inference with Auxiliary Variables](https://mlsanity.com/paper/2203.02836/) (2022)
- [Learning Proposals for Probabilistic Programs with Inference Combinators](https://mlsanity.com/paper/2103.00668/) (2021)
- [Foundation Posteriors for Approximate Probabilistic Inference](https://mlsanity.com/paper/2205.09735/) (2022)
- [Smoothness Analysis for Probabilistic Programs with Application to Optimised Variational Inference](https://mlsanity.com/paper/2208.10530/) (2022)
- [Declarative Probabilistic Logic Programming in Discrete-Continuous Domains](https://mlsanity.com/paper/2302.10674/) (2023)
- [Nonparametric Hamiltonian Monte Carlo](https://mlsanity.com/paper/2106.10238/) (2021)
- [Probabilistic Programming with Programmable Variational Inference](https://mlsanity.com/paper/2406.15742/) (2024)
- [SBI: A Simulation-Based Test of Identifiability for Bayesian Causal Inference](https://mlsanity.com/paper/2102.11761/) (2021)
- [Toward an Idiomatic Framework for Cognitive Robotics](https://mlsanity.com/paper/2111.13027/) (2021)
- [Rethinking Variational Inference for Probabilistic Programs with Stochastic Support](https://mlsanity.com/paper/2311.00594/) (2023)
- [Meta-Learning an Inference Algorithm for Probabilistic Programs](https://mlsanity.com/paper/2103.00737/) (2021)
- [Control-Data Separation and Logical Condition Propagation for Efficient Inference on Probabilistic Programs](https://mlsanity.com/paper/2101.01502/) (2021)
- [Expectation Programming: Adapting Probabilistic Programming Systems to Estimate Expectations Efficiently](https://mlsanity.com/paper/2106.04953/) (2021)
- [Unifying AI Algorithms with Probabilistic Programming using Implicitly Defined Representations](https://mlsanity.com/paper/2110.02325/) (2021)
- [Push: Concurrent Probabilistic Programming for Bayesian Deep Learning](https://mlsanity.com/paper/2306.06528/) (2023)
- [Beyond Bayesian Model Averaging over Paths in Probabilistic Programs with Stochastic Support](https://mlsanity.com/paper/2310.14888/) (2023)
- [Inference Plans for Hybrid Particle Filtering](https://mlsanity.com/paper/2408.11283/) (2024)

## Abstract (arXiv)

We describe Venture, an interactive virtual machine for probabilistic programming that aims to be sufficiently expressive, extensible, and efficient for general-purpose use. Like Church, probabilistic models and inference problems in Venture are specified via a Turing-complete, higher-order probabilistic language descended from Lisp. Unlike Church, Venture also provides a compositional language for custom inference strategies built out of scalable exact and approximate techniques. We also describe four key aspects of Venture's implementation that build on ideas from probabilistic graphical models. First, we describe the stochastic procedure interface (SPI) that specifies and encapsulates primitive random variables. The SPI supports custom control flow, higher-order probabilistic procedures, partially exchangeable sequences and ``likelihood-free'' stochastic simulators. It also supports external models that do inference over latent variables hidden from Venture. Second, we describe probabilistic execution traces (PETs), which represent execution histories of Venture programs. PETs capture conditional dependencies, existential dependencies and exchangeable coupling. Third, we describe partitions of execution histories called scaffolds that factor global inference problems into coherent sub-problems. Finally, we describe a family of stochastic regeneration algorithms for efficiently modifying PET fragments contained within scaffolds. Stochastic regeneration linear runtime scaling in cases where many previous approaches scaled quadratically. We show how to use stochastic regeneration and the SPI to implement general-purpose inference strategies such as Metropolis-Hastings, Gibbs sampling, and blocked proposals based on particle Markov chain Monte Carlo and mean-field variational inference techniques.