# Black-Box Policy Search with Probabilistic Programs

arXiv: https://arxiv.org/abs/1507.04635 · HTML: https://mlsanity.com/paper/1507.04635/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jul 2015.

## Built on by (most cited first, 4 of 4)

- [Learning Proposals for Probabilistic Programs with Inference Combinators](https://mlsanity.com/paper/2103.00668/) (2021)
- [Rethinking Variational Inference for Probabilistic Programs with Stochastic Support](https://mlsanity.com/paper/2311.00594/) (2023)
- [Bob and Alice Go to a Bar: Reasoning About Future With Probabilistic Programs](https://mlsanity.com/paper/2108.03834/) (2021)
- [MDP Planning as Policy Inference](https://mlsanity.com/paper/2602.17375/) (2026)

## Abstract (arXiv)

In this work, we explore how probabilistic programs can be used to represent policies in sequential decision problems. In this formulation, a probabilistic program is a black-box stochastic simulator for both the problem domain and the agent. We relate classic policy gradient techniques to recently introduced black-box variational methods which generalize to probabilistic program inference. We present case studies in the Canadian traveler problem, Rock Sample, and a benchmark for optimal diagnosis inspired by Guess Who. Each study illustrates how programs can efficiently represent policies using moderate numbers of parameters.